URL Analysis is a collection of techniques to determine whether a given URL is malicious or not. Generally we encounter two types of URLs i.e, Short URLs & Long URLs. In this repo we will see some tools & techniques for analyzing these URLs and figure out whether they are safe or not.
Short URLs are often used in malicious campaigns. The issue with them is that we can't simply determine where they are pointing unless we open them in some manner. Here are some tricks for converting them to original form without direct interaction.
- ZSCALER ZULU
- UNSHORTEN.IT
- URLSCAN.IO
- EXPANDURL.NET
- WHEREGOES.COM
- CHECKSHORTURL.COM
- Some services like bit.ly, goo.gl, tiny.url can be expanded by appending "
+". For e.g,bit.ly/abcdefgh+ - You can also use UrlXpand, a CLI tool I cooked after seeing Rob VandenBrink's Unshorten script.
Now that we have a long URL, we can perform some analysis on it as follows:-
- Check the WHOIS records on Databases like WHOIS, WHO.IS, or ICANN LOOKUP.
- Check the DNS records using tools like DNSDUMPSTER, or MXTOOLBOX.
- Check the site reputation & related data using Threat Intelligence platforms like VIRUSTOTAL, CISCO TALOS, PULSEDIVE, OPSWAT METADEFENDER, or CLOUDFLARE RADAR.
- Open the URL in browser sandboxes like BROWSERLING and WANNABROWSER, or SQRX with custom user agents.
- Some URLs maybe in format
https://example.com/@not-malicious.com. Here first part of URL is just for show, once a user clicks this linknot-malicious.comwill load instead ofexample.com.
- https://isc.sans.edu/diary/Taking+Apart+URL+Shorteners/28980
- https://medium.com/@joseph.pyram/9-parts-of-a-url-that-you-should-know-89fea8e11713
- https://hustlelead.medium.com/url-analysis-how-to-determine-maliciousness-f630b4e51b9e
- https://www.intezer.com/blog/malware-analysis/url-analysis-phishing-part-1/
- https://wiki.archiveteam.org/index.php?title=URLTeam#URL_shorteners
To contribute simply fork this repo, make changes and create a pull request.
If you like this repo please consider giving a ⭐.