Most sought after security packages in Python
The simple, flexible and modular design of Python combined with its powerful Rapid Application Development (RAD) support and vast libraries, makes it one of the most widely used programming languages in the field of Cyber Security. In this blog post, you will find a list of security packages curated by our interns, that help in various streams within the security realm.
1. PyTorch
PyTorch, similar to NumPy, handles a lot of computational calculations and has a vast array of features that can be used to solve various of equations. The main different with PyTorch however, is that PyTorch uses Tensors on either the CPU or the GPU. Without going into much detail, a tensor is essentially a geometric object that generalizes scalars, vectors, and matrices as an arbitrary index. Tensor calculations can take quite some time to compile because there can be so many things that the computer needs to calculate. What makes PyTorch special is that it can leverage the GPU in order to lower compile times. The main features of this package are:
- Fast computation through the utilization of GPU CUDA and Tensor cores
- Ability to change a network’s behavior through Reverse-mode auto-differentiation with zero lag or overhead
- Minimal framework overhead
- Maximum memory efficiency through custom written memory allocators
Why PyTorch for Cybersecurity?
Since the program utilizes the GPU to do intense calculations, this program would be ideal for projects that involve Big Data and need that extra horsepower in order to get results. Also, since it’s written in Python and since most Cybersecurity related projects are written in python, it would be easier to integrate this library into python projects.
Applications where the package is used:
- Data Science
- Machine Learning
- Data Analytics
Is it being actively developed?
To this day, it is being actively developed by Facebook’s Artificial Intelligence Research Group and they will soon be releasing a 1.0 version of the program. They do have a GitHub page that is being constantly updated by the hour and is very active.
2. Scapy
Scapy is a packet manipulation created by Philippe Biondi and the Scapy community, written for the Python programming language. Scapy is a sharp tool that allows the user to send, sniff, forge and decode network packets. With tools built in to improve its capabilities, it can enable the user to scan, traceroute, or attack networks. The main features of this package are:
- Packet Creation — enables the creation of custom packets
- Scanning — Identify specific details about a network, like port scanning
- Sniffing — intercepts and log the packets that flow across a particular network.
- ARP(Address Resolution Protocol) Cache Poisoning, record access points & MAC addresses
- Man in the middle attack — by writing a script in python and using Scapy tools it allows viewing a victims activities
Why Scapy for cybersecurity?
Having a toolset that allows displaying raw outputs, instead of interpretations enables security analyst to analyze the information and make independent decisions on the data. The best thing about Scapy is that it can be used as a Python library, that allows creating networking tools without going into the details of building raw packets from scratch, which considerably diminishes the size of the code.
Applications where the package is used:
- ARP cache poisoning
- Traceroutes
- Network discovery
- Access Point Spoofing
Is it being actively developed?
Scapy is principally being developed for Unix-like systems and operates best on those platforms. However, the latest version of Scapy supports Windows out-of-the-box. So you can use almost all of Scapy’s features on your Windows machine as well. Philippe Biondi is Scapy’s author. He has also written most of the documentation. Pierre Lalet, Gabriel Potter, Guillaume Valadon are the current most active maintainers and contributors. Scapy development uses the Git version control system. Project management is done with Github.
3. Scikit-learn
“Scikit-learn” is a python open source which provides many machine learning algorithms such as vector machines, random forests, gradient boosting, k-means and DBSCAN to solve classification, regression and clustering problems. It is a simple and efficient tools for data mining and data analysis. “Scikit-learn” is built on NumPy, SciPy, and matplotlib. The scikit-learn project started as scikits.learn, a Google Summer of Code project by David Cournapeau.
Why Scikit-learn for Cybersecurity?
Nowadays, Machine Learning is an important part in Cybersecurity to identify malicious behavior or malicious entities; call them hackers, attackers, malware, unwanted behavior, etc. “Scikit-learn” is a simple and efficient tools for training a big data set and analysis.
Applications where the package is used:
- Malware detection
- Network intrusion detection
- Network Traffic Analysis
Is Scikit-learn being actively developed?
Scikit-learn is considered as one of core modules of Machine Learning and Data Science field as well as the most well-designed Machine learning package. The project is currently maintained by a team of volunteers. Scikit-learn has total 6 versions. The latest version was released in July 2017. Many big companies use scikit-learn like Spotify, Booking.com, Evernote, and so on.
4. XssPy
XssPy is a fairly new python tool used for detecting Cross Site Scripting (XSS) vulnerabilities in websites. Unlike other vulnerability scanners, XssPy scans the entire website including links and sub-domains. When traversing through the website and its links, it scans every area where a malicious script can be inserted. This tool is used in through the terminal. If any vulnerabilities are found, it will be displayed on the terminal. The individual can then go directly to the vulnerable area. Some of the features of XssPy are the following:
- Checks every input field
- Traverses sub-domains
- Exact area where vulnerably is present
Why XssPy for Cybersecurity?
OWASP has listed XSS attacks as a top 10 security risk for the last few years. XSS attacks are relatively common and very dangerous. Unlike other XSS vulnerability tools I’ve found online, XssPy is free to use and has more features than other paid XSS vulnerability finders. What makes this tool perfect for cybersecurity is that it doesn’t scan just a single page, but multiple pages. This saves pen testers a lot of time since they only have to run the test once.
Is it being actively developed?
This app is being actively developed. It was last updated on January 2018. The developer has a page in his website where errors or problems can be submitted for further development.
5. Pandas
“pandas” is an open source library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language, developed by McKinney. The main features of the package are:
- Use of DataFrame object for data manipulation
- Tools to write and read data between different in-memory data structures
- Time series function, dataset reshaping, joining and merging
- Integrated handling of missing data, and more.
Why Pandas for Cybersecurity?
Leveraging Machine Learning in the field of Cybersecurity has opened arenas to improved security, threat detection and reaction to breaches. This is where Pandas comes into play with its powerful features as highlighted above and its ability to manipulate datasets required for training, modeling and analysis.
Applications where the package is used:
- Malware Analysis
- Intrusion Detection systems
- Threat Intelligence for detection of anomalies
- Network Traffic Analysis, and more.
Is it being actively developed?
Pandas is considered as one of the top Data Science and ML GitHub repositories. It’s latest release was in July 2018 having 15 other previous versions. It is currently being developed by 10 developers primarily sponsored by NumFOCUS along with Anaconda, Two Sigma as institutional partners. There are around 2000+ jobs on Indeed purely pertaining to Pandas.
6. Snort
Snort is an open source network IDS/IPS (Intrusion Detection and Prevention System) which performs detection and analysis of network traffic moving across in a more detailed way than an average firewall. IDS and IPS tools are known for analyzing traffic and comparing the packet to database of previous or known attack profiles. IDS tools alert IT staff regarding attacks, but IPS systems go a step further they block harmful traffic. A blend of the two is an essential part of a comprehensive security architecture.
Snort can be configured three different modes:
- Sniffer mode
- Packet logger mode
- Network intrusion detection mode
Why SNORT for Cybersecurity?
SNORT provides rapid response, better accuracy, and high adaptability. SNORT can protect the system from any new threats or malware through its real-time protection techniques based on the modes listed above. Since SNORT is an open source it can be updated every day to spot any new threats with new code.
Applications where the package is used:
- Semantic URL attacks
- Buffer overflows
- Server message block
- Port scans
Is it actively developed?
Snort is one of the most actively used security tools with 3,400+ companies. Snort is actively developing and modifying its set of rules in order to be a more efficient security tool to keep up with today’s threats that are constantly changing. In 2014 SNORT released OpenappID which allwows companies not to rely on layer 7 security, but now allows it to create its own application layer network security plug-in with its own specific set of rules allowing it to target specific traffic and applications.
7. GRR
GRR, or GRR Rapid Response, is an open-source incident response framework built with Python. This system is designed to allow an analyst to secure low-level information about file system and memory from one or many target machines. GRR consists of two parts, a client and a server. The client is installed on a target machine, and the server is used on the user machine to process the target data. Some of the functions of this system include:
- Live remote memory analysis.
- CPU, memory, IO usage, and imposed limits monitoring.
- Windows registry search and download capabilities.
- Enterprise hunting support, allowing for analysis of many machines.
- Automated task scheduling.
- Digital forensics artifacts collection.
Why GRR for cybersecurity?
GRR has many useful functions within the realm of cybersecurity. The main use of this tool would be for the maintenance and defense of a fleet of enterprise machines, as it is built to allow the user to collect data from hundreds, thousands, or even tens of thousands of computers at one time. This framework can act on its own or integrate with other tools in order to allow the user to rapidly detect IoCs (indicators of compromise).
Applications where package is used
- Incident response
- Live forensics
- Intrusion detection systems
- Enterprise and personal fleet monitoring
- Persistent machine access and analysis
Is it being actively developed?
Currently, GRR has a very active user-base and development team. There have been 11 version releases of GRR, with the most recent occurring 15 days ago (on June 28). Google, who began and maintains the package with the help of the community, has 5 full-time software engineers working on the project. According to the company, the project has “long-term commitment”. There are 475 forks and 7 pull requests on the project’s Github page, and around 100 positions on Indeed.com that desire experience with the GRR framework.
8. Habu
Habu is a python network hacking toolkit developed by Fabian Martinez Portantier. This tool basic functions that help with some tasks for Ethical Hacking and Penetration Testing. Most of them are related to networking, and the implementations are intended to be understandable for who wants to read the source code and learn from that. Techniques implemented in the current version are:
- ARP Poisoning
- ARP Sniffing
- DHCP Discover
- DHCP Starvation
- LAND Attack
- SNMP Cracking
- Subdomains Identification
- SYN Flooding
Why Habu for Cybersecurity
Habu gives penetration testers tools to perform cyber attacks in order to test their system. In fact, Habu comes with so many attack techniques that many hackers use. Most of the attack relates to networking such as ARP Poisoning, LAND Attack, SNMP Cracking. This is a basic, yet, powerful to most pen-testers.
Application where the package is used:
- Penetration Testing
- Read network source code
- Network Analysis (TCP, Port Scan)
- Visual Host Identification, other Identification
Is it still being developed?
Habu is constantly updating. The newest update on Github is released in June 2018. It has 15 previous version. It is currently being developed by Fabian Martinez Portantier. Habu is being used by many penetration testers, network analyst, and ethical hacker.
9. Beautiful Soup
“BeautifulSoup” is a Python library designed for quick turnaround projects like screen-scraping. It is also used for pulling data out of HTML and XML files. It is useful for getting info from websites. It was designed for the Python programming language, and parses anything you give it and does the tree traversal for the user. Developed by Leonard Richardson. The main features of the package are:
- Multiple methods to navigate, search, and modify a parse tree.
- Toolkit for dissecting a document and extracting what you need from a web application
- Simple usage allows for small amounts of code to write an application
- Automatically converts incoming documents to Unicode and outgoing documents to UTF-8.
- Allows you to try out different parsing strategies or trade speed for flexibility.
Why BeautifulSoup for Cybersecurity?
BeautifulSoup is a software used by both legitimate users working within legal bounds and malicious applications. This software is most useful in a cybersecurity toolkit when it is used to simulate the data which can be retrieved from a clients website using web scraping(Where the user has the application scan the website using available resources such as the pages source code) and screen scraping(Where the user has the application capture the image of a website and extracts data visible).
Applications where the package is used:
Is it being actively developed?
BeautifulSoup is being semi-actively developed by “Crummy”. Leonard Richardson is the owner of the webpage that holds the official information for the BeautifulSoup package. It is updated with critical bug fixes and is considered “stable” by the active developer. They also encourage users to report critical bugs. Development happens through Launchpad, There are not many contributors because development of new features has stopped.
10. NumPy
NumPy is an open source library whose main usage is supporting multidimensional arrays and matrices. This package is needed to perform scientific computing in Python. The main features are:
- Able to create and calculate N-dimensional arrays
- Numpy can integrate other languages
- Used for linear algebra calculations
- Includes random number capabilities
Why NumPy for Cybersecurity?
NumPy allows Data Analyst to paint an actual picture for others to better understand the different kinds of malware. With NumPy as a package, their python code can discover patterns within a group of malwares. That information can then lead to graphs, histograms, boxplots, etc. This is crucial in in Cybersecurity to highlight to others the main types of malware being used through data visualization.
Applications where the package is used:
- Data/ Network Traffic Analysis
- Malware Analysis
- AI
- Machine Learning
Is it being actively developed?
NumPy is one of the most important packages used by Data Science and Data Analytics to classify Malware. It’s latest version release was on June 12, 2018. There have been 39 previous versions of NumPy. On GitHub, we are able to see that there have been 18,332 commits and 663 contributors. This large number shows the number of people actively making NumPy a better/ smarter package for everyone to use.
11. Pyew
Pyew is a python tool to analyse malware. The main features of the package are:
- Supported file formats: PE, ELF, PDF and OLE2
- Searching support:
- Hexadecimal
- String (ASCII and Unicode)
- Regular expression
- Disassembly (Intel 16, 32 and 64 bits)
- URL extractor and checker.
Why Pyew for Cybersecurity?
- Malware Analysis in batch mode is possible.
- When analyzing PDF malware exploits, typically to see hidden JavaScript code, don’t need to extract and decode each file one by one
Is it being actively developed?
The latest stable version, Version 2.3, was released on 01–13–2014. However, the last update of the github wiki homepage was updated on Nov 22, 2016.
12. PyDbgEng
Python Debugging Engine is a powerful windows debugging tool mostly used to debug applications or to test vulnerabilities on applications. The main features are:
- User mode Debugging
- Kernel mode Debugging
- Version x80 and x64 support
- Soft and HW breakpoints
Why PyDbgEng for CyberSecurity?
This debugging tool gives the opportunity to developers to discover vulnerabilities to their applications which they can then fix. It stresses the application looking for a place where the application does not respond to, then tells the user where exactly that bug is at. It can be an important tool for cybersecurity because of its precise bug finder.
Applications where PyDbgEng is used?
- Pen-Testing
- Fault Injection
- Application Fuzzing
- Automatic Executable Unpacking
Is it being actively developed?
Python Debugging Engine isn’t being developed as of today. The earliest known version of PyDbgEng was in 2007 which of course was a while ago but on the contrary the last known update was 2013. Nevertheless, PythDbgEng is a very useful tool for developers in those years because it was quite advanced.
13. Androguard
Androguard is a useful tool created by Anthony Desnos using python. It can be used to play with android files and applications. It is available for Linux, Windows, and OSX systems. The main features:
- Reverse Engineer android applications
- Check if applications have been pirated
- Review for malicious intent
- Investigate malware files
- Study goodware files
Why Androguard for Cybersecurity?
Part of Cybersecurity is the forensics department, where they study malware, goodware and see the differences and how they act. Reverse Engineering in a safe pace is a very important part of this process. It is necessary to analyze Android-Manifest for permissions and activities, unpacking of android applications to get all files inside, and analyze the generated code. All of this can be done using the androguard tool.
Applications where Androguard is used:
- Anubis (Andrubis)
- Virustotal
- Androwarn
- Googleplay-api
- MalloDroid
Is Androguard being actively developed?
Androguard is actively being developed because there were minor changes to their newest version on May 1, and before that it was in April 28 of this year. The documentation for their newest release 3.2.0 was written in June 29 of this year.
It doesn’t stop here! The list of the packages currently being used in cyber security applications is exhaustive and the ones highlighted here are a few of the majorly used packages in the industry.
