Easily turn single threaded command line applications into a fast, multi-threaded application with CIDR and glob support.

codingo, updated 🕥 2022-11-03 16:41:55

Interlace

Easily turn single threaded command line applications into a fast, multi-threaded application with CIDR and glob support.

Python 3.2|3.6 License Twitter Twitter

Reviews / Howto Guides

Interlace: A Tool to Easily Automate and Multithread Your Pentesting & Bug Bounty Workflow Without Any Coding

Table of Contents

Setup

Install using:

bash $ python3 setup.py install Dependencies will then be installed and Interlace will be added to your path as interlace.

Usage

| Argument | Description | |------------|--------------------------------------------------------------------------------------------------------------| | (stdin) | Pipe target lists from another application in comma-delimited format, CIDR notation, or as an individual host| | -t | Specify a target or domain name either in comma-delimited format, CIDR notation, or as an individual host | | -tL | Specify a list of targets or domain names | | -e | Specify a list of targets to exclude either in comma-delimited format, CIDR notation, or as an individual host | | -eL | Specify a list of targets to exclude | | -threads | Specify the maximum number of threads to run at any one time (DEFAULT:5) | | -timeout | Specify a timeout value in seconds for any single thread (DEFAULT:600) | | -c | Specify a single command to execute over each target or domain | | -cL | Specify a list of commands to execute over each target or domain | | -o | Specify an output folder variable that can be used in commands as _output_ | | -p | Specify a list of port variable that can be used in commands as _port_. This can be a single port, a comma-delimited list, or a range using dash notation | | -pL | Specify a list of proxies | | --proto | Specify protocols that can be used in commands as _proto_ | | -rp | Specify a real port variable that can be used in commands as _realport_ | | -random | Specify a directory of files that can be randomly used in commands as _random_ | | --no-bar / --sober | If set then progress bar be stripped out | | --no-cidr | If set then CIDR notation in a target file will not be automatically be expanded into individual hosts | | --no-color | If set then any foreground or background colours will be stripped out | | --silent | If set then only important information will be displayed and banners and other information will be redacted | | -v | If set then verbose output will be displayed in the terminal |

Further information regarding ports

| Example | Notation Type | |---------|----------------------------------------------------------| | 80 | Single port | | 1-80 | Dash notation, perform a command for each port from 1-80 | | 80,443 | Perform a command for both port 80, and port 443 |

Further information regarding targets

Both -t and -tL will be processed the same. You can pass targets the same as you would when using nmap. This can be done using CIDR notation, dash notation, or a comma-delimited list of targets. A single target list file can also use different notation types per line.

Alternatively, you can pass targets in via STDIN and neither -t or -tL will be required.

Variable Replacements

The following variables will be replaced in commands at runtime:

| Variable | Replacement | |-----------|-------------------------------------------------------------------------| | _target_ | Replaced with the expanded target list that the current thread is running against | | _cleantarget_ | Replaced with target cleanded from http:// or https:// | | _host_ | Works the same as _target_, and can be used interchangeably | | _output_ | Replaced with the output folder variable from Interlace | | _port_ | Replaced with the expanded port variable from Interlace | | _realport_ | Replaced with the real port variable from Interlace | | _proxy_ | Replaced with the proxy list from Interlace | | _random_ | Replaced with the randomly chosen file from Interlace |

Advanced Command File Usage

Interlace also makes the use of two additional features for controlling execution flow within a command file: _blocker_ and _block:<name>_. Blockers prevent execution of commands listed after them, until all commands before them have completed, and blocks can be used to force sequential execution of commands listed within a block, for a target.

These are run on a per-target level. If there are threads available and a blocker is in the way for the current target, Interlace will start commands from the next target within a target list in order to maximise efficiency.

Using these features will allow you to control the execution flow for individual targets more directly in order to prevent commands from running out of order.

Blocker

Blockers prevent anything below them from executing until all commands above them have completed (for the currently active host). For example, in the following:

mkdir -p _output_/_target_/scans/ _blocker_ nmap _target_ -oA _output_/_target_/scans/_target_-nmap

The use of a blocker here prevents nmap from running on a target before the base folder structure has been created, preventing nmap from throwing an exception.

Blocks

Blocks force everything within them to run sequentially. You can also use multiple blocks per command file. For example, in the following:

_block:nmap_ mkdir -p _target_/output/scans/ nmap _target_ -oN _target_/output/scans/_target_-nmap _block:nmap_ nikto --host _target_ In this example, the block would run the same as before, but assuming the thread count is high enough then nikto would begin to run immediately, passing results back to the terminal (whilst nmap and file creation happened in the background).

Usage Examples

Run Nikto Over Multiple Sites

Let's assume that you have a file targets.txt that has the following contents:

bugcrowd.com hackerone.com You could use Interlace to run over any number of targets within this file using: bash ➜ /tmp interlace -tL ./targets.txt -threads 5 -c "nikto --host _target_ > ./_target_-nikto.txt" -v ========================================================================= Interlace v1.0 by Michael Skelton (@codingo_) & Sajeeb Lohani (@sml555_) ========================================================================= [14:33:23] [THREAD] [nikto --host hackerone.com > ./hackerone.com-nikto.txt] Added to Queue [14:33:23] [THREAD] [nikto --host bugcrowd.com > ./bugcrowd.com-nikto.txt] Added to Queue This would run Nikto over each host and save to a file for each target. Note that in the above example since we're using the > operator, the results won't be fed back to the terminal; however this is desired functionality as otherwise we wouldn't be able to attribute which target Nikto results were returning for.

For applications where you desire feedback, simply pass commands as you normally would (or use tee).

Run Nikto Over Multiple Sites and Ports

Using the above example, let's assume you want independent scans to be run for both ports 80 and 443 for the same targets. You would then use the following:

➜ /tmp interlace -tL ./targets.txt -threads 5 -c "nikto --host _target_:_port_ > ./_target_-_port_-nikto.txt" -p 80,443 -v ========================================================================= Interlace v1.0 by Michael Skelton (@codingo_) & Sajeeb Lohani (@sml555_) ========================================================================= [14:33:23] [THREAD] [nikto --host hackerone.com:80 > ./hackerone.com-nikto.txt] Added to Queue [14:33:23] [THREAD] [nikto --host bugcrowd.com:80 > ./hackerone.com-nikto.txt] Added to Queue [14:33:23] [THREAD] [nikto --host bugcrowd.com:443 > ./bugcrowd.com-nikto.txt] Added to Queue [14:33:23] [THREAD] [nikto --host hackerone.com:443 > ./hackerone.com-nikto.txt] Added to Queue

Run a List of Commands against Target Hosts

Often with penetration tests, there's a list of commands you want to run on nearly every job. Assuming that list includes testssl.sh, nikto, and sslscan, you could save a command list with the following in a file called commands.txt:

nikto --host _target_:_port_ > _output_/_target_-nikto.txt sslscan _target_:_port_ > _output_/_target_-sslscan.txt testssl.sh _target_:_port_ > _output_/_target_-testssl.txt If you were then given a target example.com, you could run each of these commands against this target using the following: bash interlace -t example.com -o ~/Engagements/example/ -cL ./commands.txt -p 80,443 This would then run nikto, sslscan, and testssl.sh for both port 80 and 443 against example.com and save the files into your engagements folder.

CIDR notation with an application that doesn't support it

Interlace automatically expands CIDR notation when starting threads (unless the --no-cidr flag is passed). This allows you to pass CIDR notation to a variety of applications:

To run a virtual host scan against every target within 192.168.12.0/24 using a direct command you could use: bash interlace -t 192.168.12.0/24 -c "vhostscan _target_ -oN _output_/_target_-vhosts.txt" -o ~/scans/ -threads 50 This is despite VHostScan not having any in-built CIDR notation support. Since Interlace expands the notation before building a queue of threads, VHostScan for all intents is only receiving a list of direct IP addresses to scan.

Glob notation with an application that doesn't support it

Interlace automatically expands glob ranges when starting threads. This allows you to pass glob ranges to a variety of applications:

To run a virtual host scan against every target within 192.168.12.* using a direct command you could use: bash interlace -t 192.168.12.* -c "vhostscan _target_ -oN _output_/_target_-vhosts.txt" -o ~/scans/ -threads 50 Yet again, VHostScan does not have any inbuilt glob range format support.

Dash (-) notation with an application that doesn't support it

Interlace automatically expands dash ranges when starting threads. This allows you to pass glob ranges to a variety of applications:

To run a virtual host scan against every target within 192.168.12.1-15 using a direct command you could use: bash interlace -t 192.168.12.1-15 -c "vhostscan _target_ -oN _output_/_target_-vhosts.txt" -o ~/scans/ -threads 50 Yet again, VHostScan does not have any inbuilt dash range format support.

Threading Support for an application that doesn't support it

Run a virtual host scan against each host in a file (target-lst.txt), whilst also limiting scans at any one time to 50 maximum threads.

This could be done using a direct command: bash interlace -tL ./target-list.txt -c "vhostscan -t _target_ -oN _output_/_target_-vhosts.txt" -o ~/scans/ -threads 50

Or, alternatively, to run the same command as above, but using a command file: bash interlace -cL ./vhosts-commands.txt -tL ./target-list.txt -threads 50 -o ~/scans This presumes that the content of the command file is: vhostscan -t $target -oN _output_/_target_-vhosts.txt This would output a file for each target in the specified output folder. You could also run multiple commands simply by adding them into the command file.

Exclusions

Interlace automatically excludes any hosts provided when specified via the -e or -eL arguments. These arguments are also compatible with the range notations mentioned above (CIDR, glob, and dash)

To run a virtual host scan against every target in the CIDR range 192.168.12.0/24 but not for the targets in the range 192.168.12.0/26, using a direct command, you could use: bash interlace -t 192.168.12.0/24 -e 192.168.12.0/26 -c "vhostscan _target_ -oN _output_/_target_-vhosts.txt" -o ~/scans/ -threads 50

Run Nikto Using Multiple Proxies

Using the above example, let's assume you want independent scans to be via different proxies for the same targets. You would then use the following:

➜ /tmp interlace -tL ./targets.txt -pL ./proxies.txt -threads 5 -c "nikto --host _target_:_port_ -useproxy _proxy_ > ./_target_-_port_-nikto.txt" -p 80,443 -v

Authors and Thanks

Originally written by Michael Skelton (codingo) and Sajeeb Lohani (sml555) with help from Charelle Collett (@Charcol0x89) for threading refactoring and overall approach, and Luke Stephens (hakluke) for testing and approach.

Contributions

Contributions to this project are very welcome. If you're a newcomer to open source and would like some help in doing so, feel free to reach out to us on Twitter (@codingo_) / (@sml555_) and we'll assist wherever we can.

Issues

Multithreading doesn't work on Windows

opened on 2022-10-06 11:38:32 by mkey-hu

On Widows OS tasks are executed in a serialized way

``` shell interlace -t a,b,c -c "timeout /t 10 & echo target" ===================================================== Interlace v1.9.6 by Michael Skelton (@codingo_) & Sajeeb Lohani (@sml555_) ===================================================== 0%| | 0/3 [00:00<?, ?it/s][13:34:29] [THREAD] [timeout /t 10 & echo a] Added to Queue

Waiting for 10 seconds, press a key to continue . 0 a

33%|████████████████████████████ | 1/3 [00:09<00:19, 9.71s/it][13:34:39] [THREAD] [timeout /t 10 & echo c] Added to Queue

Waiting for 10 seconds, press a key to continue . 0 c

67%|████████████████████████████████████████████████████████ | 2/3 [00:19<00:04, 4.85s/it][13:34:49] [THREAD] [timeout /t 10 & echo b] Added to Queue

Waiting for 10 seconds, press a key to continue . 0 b

100%|████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:29<00:00, 9.92s/it] ```

Interlace parsing break when encountered with comma

opened on 2022-09-13 08:06:00 by tarunkant

Hello, I noticed that Interlace could not parse URLs when it contains commas.

Example test file:

http://example.com/eName,p=123?l.host+l.path=123 http://example.com/eNamep=123,asds&asd1=asd,asddd

Command:

interlace -tL test -c 'echo "_host_"'

Output:

``` http://example.com/eNamep=123
p=123?l.host+l.path=123
asds&asd1=asd
http://example.com/eName
asddd
````

Expected Output:

http://example.com/eName,p=123?l.host+l.path=123 http://example.com/eNamep=123,asds&asd1=asd,asddd

Add more "Variable Replacement"

opened on 2022-08-26 12:57:51 by MarcoZufferli None

Python Error while running

opened on 2022-07-23 02:13:52 by rasiras

Getting this error while running interlace

File "/usr/local/bin/interlace", line 33, in <module> sys.exit(load_entry_point('Interlace==1.9.6', 'console_scripts', 'interlace')()) File "/usr/local/lib/python3.9/dist-packages/Interlace-1.9.6-py3.9.egg/Interlace/interlace.py", line 24, in main File "/usr/local/lib/python3.9/dist-packages/Interlace-1.9.6-py3.9.egg/Interlace/lib/core/input.py", line 381, in parse File "/usr/lib/python3.9/argparse.py", line 1830, in parse_args args, argv = self.parse_known_args(args, namespace) File "/usr/lib/python3.9/argparse.py", line 1863, in parse_known_args namespace, args = self._parse_known_args(args, namespace) File "/usr/lib/python3.9/argparse.py", line 2072, in _parse_known_args start_index = consume_optional(start_index) File "/usr/lib/python3.9/argparse.py", line 2012, in consume_optional take_action(action, args, option_string) File "/usr/lib/python3.9/argparse.py", line 1924, in take_action argument_values = self._get_values(action, argument_strings) File "/usr/lib/python3.9/argparse.py", line 2455, in _get_values value = self._get_value(action, arg_string) File "/usr/lib/python3.9/argparse.py", line 2488, in _get_value result = type_func(arg_string) File "/usr/local/lib/python3.9/dist-packages/Interlace-1.9.6-py3.9.egg/Interlace/lib/core/input.py", line 427, in <lambda> File "/usr/local/lib/python3.9/dist-packages/Interlace-1.9.6-py3.9.egg/Interlace/lib/core/input.py", line 33, in check_positive AttributeError: 'ArgumentParser' object has no attribute 'ArgumentTypeError'

any suggestions to fix this will be appreciated

``` └──╼ $python --version Python 3.9.2

```

Interlace dont like latest python

opened on 2022-05-16 07:45:05 by shifty0g

Not sure if it likes new version of python

``` uname -a Linux kali 5.16.0-kali7-amd64 #1 SMP PREEMPT Debian 5.16.18-1kali1 (2022-04-01) x86_64 GNU/Linux

python3 --version
Python 3.10.4 ```

Steps to replicate

Literally git clone this repo

cd Interlace python3 setup.py install

when i try to then load interlace it dies

``` python3 interlace.py

Traceback (most recent call last): File "/usr/local/bin/interlace", line 33, in sys.exit(load_entry_point('Interlace==1.9.6', 'console_scripts', 'interlace')()) File "/usr/local/bin/interlace", line 25, in importlib_load_entry_point return next(matches).load() File "/usr/lib/python3.10/importlib/metadata/init.py", line 171, in load module = import_module(match.group('module')) File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/interlace.py", line 6, in File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/lib/core/output.py", line 4, in File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/init.py", line 11, in File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/codes.py", line 4, in ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/init.py) ```

any ideas what im doing wrong here. i have reverted snapshot and tried on a fresh kali vm but no go. seems like interlace dont play ball with the latest python

Installation errors on Mac

opened on 2022-05-12 13:19:48 by Teicu

Hello,

Does anyone know why I get this error and can't install it?

Thanks

python3 setup.py install

Traceback (most recent call last): File "setup.py", line 19, in <module> setup( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 86, in setup _install_setup_requires(attrs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires dist = MinimalDistribution(attrs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 57, in __init__ super().__init__(filtered) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/dist.py", line 460, in __init__ for ep in metadata.entry_points(group='distutils.setup_keywords'): File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 999, in entry_points return SelectableGroups.load(eps).select(**params) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 449, in load ordered = sorted(eps, key=by_group) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 996, in <genexpr> eps = itertools.chain.from_iterable( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/_itertools.py", line 16, in unique_everseen k = key(element) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 931, in _normalized_name return self._name_from_stem(stem) or super()._normalized_name File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 600, in _normalized_name return Prepared.normalize(self.name) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 855, in normalize return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_') File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/re.py", line 210, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes-like object

Releases

Version 1.6 2019-08-19 12:50:11

New Features and Improvements

  • Introduced _random_ in #58
  • Introduced _blocker_ and _block:xyz_ in #60

Bug Fixes

  • Fix Arguments.output bug introduced in #57 in #61
  • Permissions issues bugfix in #57
  • Fixed bug with long commands and de-duplication in #49
  • Various progress bar bugfixes in #45

Version 1.5.1 2019-06-04 05:08:19

New Features and Improvements

  • Added progress bar in #42
  • Added Proxy List support in #40
  • Refactor for speed improvements over exceptionally large lists in #30

Bug Fixes

  • Fix Case-sensitive file calls on subprocess.call() in #38
  • Fix character stripping bug in #33
  • Fix progress bar not sticking to terminal in #44

Version 1.3.5 2019-04-30 09:48:16

Bug Fixes

  • Fixed Multi Threading not working properly in OS/X in #37

Version 1.3.4 2019-04-12 00:24:49

New Features and Improvements

  • Added Stdin Support in #22
  • Added _proto_ variable in #23
  • Significant speed improvements with very large lists in #30

Bug Fixes

  • Fixed empty port, exclusion or domain name throwing exception in #18
  • Fixed empty target list bug in #35
  • Fixed broken setup when client only using Python 3.7 in #21
  • Added checks for output folder existence in #23
  • Fixed realports replacement not consistently working in #27
  • Removed [thread] outputs when -silent flag is used in #29
  • Fixed trimmed spaces bug in #33

Version 1.1 2019-01-09 10:58:30

Features

  • added exclusion list feature by request in #13 and #15

Bugfixes

  • Colour ordering fix (minor) in #3

First Public Release 2019-01-06 10:55:41

First public release of Interlace.

Michael Skelton

Senior Director of Security Operations @ Bugcrowd

GitHub Repository

linux service-enumeration penetration-testing security security-tools enumeration multithreading hacking hacking-tool bugbounty cidr-notation thread oscp oscp-tools