By correctly identifying the underlying technologies running on a site, pentesters gain a significant advantage in preparing an attack. Whether you are testing the defense of a large company or playing the latest CTF, it is crucial to find out which technologies are used on a website.
Knowing the technology and codebase used to create a website can speed things up by eliminating potential attacking methods or exploits that we know will not work. It can also reduce the likelihood your intrusion will be detected by triggering fewer alarms.
Today we will explore a tool called WebTech to bring these technologies to light.
WebTech is an open-tech system. Source Python tool for identifying Web technologies. You can use it in several ways, including as a command-line tool, as a Burp Suite extension, and as a Python library for scripts.
Other tools are available for this purpose, such as: B. Wappalyzer used browser extension or other online alternatives such as W3Techs or these of Pentest tools. WebTech, however, has an extremely modular and easy-to-use method, especially with respect to Python scripts.
The reconnaissance phase is critical to the ethical hacking or penetration test process. The old saying "measure twice, cut once" applies here.
Installation and Basic Usage
In Order To use WebTech, we first need to install it. While you can use WebTech on any operating system that supports Python, I'll show you how it works in Kali Linux (or any other Debian-based distribution).
First, make sure that Python is installed on your device – we can check with the command which :
~ # which Python / usr / bin / python
If you do not see any output of it, install it with the package manager:
~ # apt-get install python
Then install pip, a package management system for Python, with the following command :
~ # apt-get install python-pip
Now we can finally install WebTech:
~ # pip install webtech Collect Webtech Download from https://files.pythonhosted.org/packages/a7/66/3bd231369ca661e76fa863546c2d7d8c73fd214fc018dcee37ff32a368d8/webtech-1.2.7.tar.gz (103kB) 100% | ████████████████████████████████ | 112 kB 1.5 MB / s Requirement already met: Requirements in /usr/lib/python2.7/dist-packages (by webtech) (2.21.0) Building wheels for collective packaging: webtech Run Setup.py bdist_wheel for webtech ... done Filed in: /root/.cache/pip/wheels/36/0d/d6/67a0bbbfd449ecb578cac82c098668ef032dbd513640257c94 Webtech successfully established Install collected packages: webtech Successfully Installed webtech 1.2.7
Simply enter webtech into the terminal to display its usage and options:
~ # webtech No URL (s) given! Use: webtech [options] options: --version Display and exit the version number of the program -h, --help View and exit this help message -u URLs, --urls = URLs to be scanned --urls-file = URLS_FILE, --ul = URLS_FILE URL (s) list file for scanning --user-agent = USER_AGENT, --ua = USER_AGENT Use this user agent - random user agent, --rua Use a random user agent --database-file = DATABASE_FILE, --db = DATABASE_FILE custom database file --json, --oj outputs a json-coded report --grep, --og outputs a grepable report --update -db, --udb forces the update of remote database files --timeout = TIMEOUT maximum timeout for scrape requests
The simplest use of the tool is to specify a URL to scan with the flag -u :
~ # webtech -u https: // null-byte.wonderhowto.com/ Destination URL: https://null-byte.wonderhowto.com/ Detected technologies: - jQuery 1.7 - Google Font API - ComScore The following interesting custom headers were found: - Server: WonderHowTo - X-UA compatible: IE = Edge, Chrome = 1 - X server name: APP02
Here we see the detected technologies used by the site, as well as some interesting headers. This information can be helpful in preparing an attack, as reducing unnecessary additional variables can dramatically reduce the time required to succeed.
With this tool, we can also specify a custom user agent that can sometimes be used to search the site for different answers, depending on how it's set up. User agents identify the browser and the operating system against the web server and are sent as a text string in HTTP headers.
Find a database of virtually all user agents you can think of to help with the exam.  Find the user program under test and add it to your URL – ua = & # 39; [USERAGENTCODE] & # 39; instead of USERAGENTCODE in this string. This is what it looks like:
~ # webtech -u https://null-byte.wonderhowto.com/ --ua = & # 39; Mozilla / 5.0 (Linux; Android 6.0.1; SM-G920V Build / MMB29K ) AppleWebKit /537.36 (KHTML, like Gecko) Chrome / 52.0.2743.98 Mobile Safari / 537.36 & # 39; Destination URL: https://null-byte.wonderhowto.com/ Detected technologies: - jQuery 1.7 - Google Font API - ComScore The following interesting custom headers were found: - Server: WonderHowTo - X-UA compatible: IE = Edge, Chrome = 1 - X server name: APP02
Here we set the user agent to mimic a Samsung Galaxy S6, although we did not get different results. For example, if an error occurs in a particular browser, the server will respond differently depending on the user agent.
We could also set a random user agent if we so wished. -random-user-agent flag randomly selecting a user agent to use.
Using the WebTech command line is definitely helpful when it comes to identifying the underlying technologies of a website used in scripts.
Scripting with WebTech
You can use WebTech in any Python script by simply importing the library. This flexibility is, in my opinion, the most useful feature of the tool, as you can integrate it into any other script that depends on what a website is based on.
For example, WebTech could be used as part of a larger fingerprint script alongside port scans and enumeration of services. It's also useful in exploit scripts, where the details of the exploit change slightly depending on the platform.
Let's create a quick demo script to show it in action. Create your Python file with a text editor of your choice. In this case, I use nano because it is simple:
~ # nano scan.py
The first line should contain the script To perform the operation, go to our Python binary show:
#! / usr / bin / python
Next we need an import statement to import the WebTech library:
Then we can create a new instance with the variable wt :
wt = webtech.WebTech ()
And start a scan of the desired URL (I also had to set a short timeout, otherwise it would not work) and save this in the variable results :
results = wt.start_from_url (& # 39; https: //null-byte.wonderhowto.com/' ;, timeout = 1)
Finally, we can print the results of the scan on the Screen Displayed:
The final script should look like this:
#! / usr / bin / python import webtech wt = webtech.WebTech () results = wt.start_from_url (& # 39; https: //null-byte.wonderhowto.com/' ;, timeout = 1) Print Results
We can now execute our script with the command python set up above:
~ # python scan.py Destination URL: https://null-byte.wonderhowto.com/ Detected technologies: - jQuery 1.7 - Google Font API - ComScore The following interesting custom headers were found: - Server: WonderHowTo - X-UA compatible: IE = Edge, Chrome = 1 - X server name: APP03
Here we see that we got the same results as before. Remember, this was just a simple proof-of-concept – we could make this much more robust if we wanted to.
This article describes WebTech, a Python site technology identification tool.
We saw how it can be installed with pip and easily run from the command line. We also examined its true power, the ability to be used as a Python library, by writing our little script. WebTech makes it easy to get to know your target during the reconnaissance phase and gives you the advantage of planning an attack.