![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQXMX2jdUowJknKy3Tk6D0JKin4_rnTN9iVKouL6JHgL1DsIG-lIT09mx7cauHr-A04dbi_oi-o_M5ZMeh2QPOgSa9kvLWNeNeTDR4b41huitZIbIcCxCmyB4XSKmbLuRaYTWIGyk2wgvS/s640/TheTHE.png) TheTHE is an environment intended to help analysts and hunters over the early stages of their work in an easier, unified and quicker way. One of the major drawbacks when dealing with a hunting is the collection of information available on a high number of sources, both public and private. All this information is usually scattered and sometimes even volatile. Perhaps at a certain point there is no information on a particular IOC (Indicator of Compromise), but that situation may change within a few hours and become crucial for the investigation. Based on our experience on Threat Hunting, we have created a free and open source framework to make the early stages of the investigation simpler from: * Server-client architecture. Investigation may be shared among your team. * APIkeys are stored in a database and may be shared by a team from a single point. * Results are cached; so not repeated API calls are used. * Better feeds your Threat Intelligence Platform. TheTHE allows to better perform a prior investigation of your assets. * Easy plugins: Whatever you need, it is easily embedded within the system. * Ideal for SOCs, CERTS or any team. * Automation of tasks and searches. * Rapid API processing of multiple tools. * Unification of information in a single interface, so that screenshots, spreadsheets, text files, etc. are not scattered. * Enrichment of collected data. * Periodic monitoring of a given IOC in case new information or related movements appear. TheTHE has a web interface where the analyst starts its work by entering IOCs that will be sent to a backend, where the system will automatically look up for such resource on the various configured platforms in order to obtain unified information from different sources and access related reports or data existing on them. Furthermore, any change in the resources to be analyzed will be monitored. Everything is executed on a local system, without needing to share information with third parties until such information is not organized, linked, complete and synthesized. This allows that, in case the information must be analyzed later on any other platform (such as a Threat Intelligence Platform), it can be done in the most enriching possible manner. **A complete docker image is scheduled to be released soon, meanwhile you can play with the dev enviroment** **Install (development enviroment)** * git clone ... **Creating a Python virtual enviroment** * pip install venv * python3 -m venv venv **Pull docker images** * docker-compose build **Install node packages** * cd frontend and npm install **Running (development enviroment)** Activate the Python virtual enviroment (from the project root): source venv/bin/activate You can open two terminal sessions (with both virtual enviroments activated) or just put the following two commands in background. Backend: watchmedo auto-restart -d tasks -p '*.py' -- celery -A tasks.tasks:celery_app worker -l info gunicorn server.main:app --reload Frontend (you won't be able to operate until a user is added to the system): cd frontend npm run serve **Setting up the initial account** You must open a session in the MongoDB (remember is a docker container): docker exec -it thethe_mongo_1 /bin/bash Create a collection (Mongodb) called "thethe" (of course, you MUST change docker-compose default passwords): mongo -u MONGO_INITDB_ROOT_USERNAME -p MONGO_INITDB_ROOT_PASSWORD Now, inside mongo shell: use thethe Create "users" and YOUR first user: db.users.insert_one({"name": YOUR_NAME, "password": HASH}) _hash_ is (assuming python env is activated and requirements.txt are installed) gotten from a python interactive interpreter: from passlib.apps import custom_app_context as pwd_context pwd_context.encrypt("yourpassword") Copy the hash into the value of password field. Done. **How to create a plugin for THETHE** Plugins affect one or more resources. It may have a frontend counterpart written as a Vue (javascript) component. Plugins must registers itself to describe what they do, etc. When the plugin is launched it may be queued in a Celery queue for background processing (or not, depending in the kind of tasks, actions, etc.) A plugin does have the following structure: **Python plugin** A plugin in Python is just for describe the actions, description, name, category, etc. It must inherit from **plugin_base/Plugin** The plugin **MUST** be located in **server/plugins/** folder. Below is an exrtract of **server/plugins/geoip.py** RESOURCE_TARGET = [ResourceType.IPv4] # Plugin Metadata {a decription, if target is actively reached and name} PLUGIN_DESCRIPTION = "Use a GeoIP service to geolocate an IP address" PLUGIN_IS_ACTIVE = False PLUGIN_NAME = "geoip" class Plugin: description = PLUGIN_DESCRIPTION is_active = PLUGIN_IS_ACTIVE name = PLUGIN_NAME def __init__(self, resource, project_id): self.project_id = project_id self.resource = resource def do(self): resource_type = self.resource.get_type() try: to_task = { "ip": self.resource.get_data()["address"], "resource_id": self.resource.get_id_as_string(), "project_id": self.project_id, "resource_type": resource_type.value, "plugin_name": Plugin.name, } geoip_task.delay(**to_task) except Exception as e: tb1 = traceback.TracebackException.from_exception(e) print("".join(tb1.format())) **Celery task** Finally, the task is introduced as a Celery function which gives us async results. It **SHOULD** be located into **tasks/tasks.py** file with the proper Celery decorators @celery_app.task def geoip_task(plugin_name, project_id, resource_id, resource_type, ip): try: query_result = geoip(ip) if not query_result: return # TODO: See if ResourceType.__str__ can be use for serialization resource_type = ResourceType(resource_type) resource = Resources.get(resource_id, resource_type) resource.set_plugin_results( plugin_name, project_id, resource_id, resource_type, query_result ) except Exception as e: tb1 = traceback.TracebackException.from_exception(e) print("".join(tb1.format())) **Celery subtask** You **MAY** split the core functionality in a external module located into the **tasks/subtasks** folder. def geoip(ip): try: URL = f"http://api.ipstack.com/{ip}?access_key={API_KEY}&format=1" response = urllib.request.urlopen(URL).read() return json.loads(response) except Exception as e: tb1 = traceback.TracebackException.from_exception(e) print("".join(tb1.format())) return None **Vue component** Finally, the frontend **MIGHT** know hot to represent the results as a view. As the system is designed to support dynamic load of plugins all the Vue components are loaded under demand via the **DynamicComponent** component. Plugins must have an entry into the **frontend/src/components/templates/ /index.vue** Here you must deal with the plugins results presentation. As an example (taking GeoIp plugin): **Download Thethe**