Share with your friends


Open source intelligence techniques & commentary

Automating OSINT

This blog is all about collecting, and storing open sources of information. In particular how do we automate some of these tasks so that we can spend more time doing what humans do best: analysis. The web is full of content, APIs and is rich for data analysis. But where do we start?

Too often I see people attacking this problem by collecting everything and then trying to sort it out later. This falls into the “big data” trap of somehow collecting vast amounts of information and then implementing some magical algorithm to sort through it all later. Producing pie graphs, fancy network diagrams and other fluff does not equate to good OSINT technique, but it sure sells product. Don’t get me wrong, I love visualizations as much as the next guy, but you have to understand what those visualizations mean, and better yet, how to tweak them to suit your target. The “big data” approach is fine if you have a PhD. in computer science and a shit pile of time and money.

That is not me. And I doubt it is you.

Herein lies the problem. Most of the people who identify that OSINT is a magical beast with a pot of gold tucked under its nethers, are those people who have hundreds (if not thousands) of times performed manual OSINT. They “get it” because, when done properly, it can yield amazing and surprising results. They have saved HTML pages by hand, used macros in Microsoft Word, created spreadsheets till their eyes bleed and abused news readers and Twitter analysis apps until their brains hurt. I have seen it all.

Strangely enough, it is these people who also are scared to write code. The thought of “computer programming” to a traditional detective, or journalist is a scary endeavour. I am here to tell you (assuming you are one of these people) that it doesn’t have to be like this. In fact my course will break down that mental barrier and teach you all of the really amazing tricks you can learn and apply with a little code. It is guaranteed to put targeted data at your fingertips for your next case, your next story or to help enhance your ability to make business decisions.

I will track interesting tools of the trade in this blog and update fellow readers on new course modules as they are implemented. I’ll likely share some tricks along the way and point out some amazing people doing really neat things with OSINT of all kinds in other spots around the web.