Securing the Digital World

The Realm of Threat Intelligence - Journey from the past into an Advanced SOC

Sep 27, 2016 | by David Gray |

Using Intelligence to gather information on your adversary is not a new concept, Military and Government Agencies have been involved gathering information to use against their opponents since the days of Sun-Tzu and Chanakya. Cyber Intelligence has also been the domain for Government agencies like the UK's GCHQ and the US's NSA for many years; long before the general public was aware of threats from Cyber Actors or APT groups. Cyber Threat Intelligence itself poses a challenge in that no single organisation has access to an adequate pool of relevant information to accurately gauge the situational awareness of the threat landscape they face. For too long, businesses have been ignorant of these threats. Fortunately over the last few years this situation has started to change.

Government Agencies keen to protect their countries key National Infrastructure and Defense contractors started schemes to share a degree of this Intelligence. These organisations wanted to start working together sharing their experiences and knowledge. It was an established fact that one Defense contractor would be targeted by the same Threat Actor who had attacked an industry peer the previous week, yet nothing was being passed to potential targets. In working together the Defense Contractor companies were able to identify threats they had been subjected to and notified each other thus allowing a larger pool of threat information to be gathered and enhancing the detection and blocking strategies. Over the last 10 years these small groups have expanded outside of the Defense industry to encapsulate a wider pool of companies, their analysts and their Threat Picture.

Open Source tools and their support community have also driven forward sharing of Intelligence. Martin Roesch created SNORT back in 1998 which to this day allows analysts to create and import their own bespoke IDS signatures. In a similar fashion Yara, OpenIOC and other formats have allowed SOC staff to share information. However this sharing of information has always been done on an ad-hoc and unstructured basis with Google being used more than once to identify these Signatures and IOC's (which of course is also available to the attackers themselves allowing them to change their TTP's (Tactics, Tools & Procedures)).

There have been many attempts to leverage these Threat Intelligence artefacts into one industry standard. Mandiant lead the way with the open source OpenIOC standard, however there was little integration of the standard from the security community outside of Mandiant's own technologies. More recently STIX (Structured Threat Information eXpression) has come to the fore as a collaborative community-driven effort to define and develop a standardised language to represent structured cyber threat information. This has been coupled with TAXI (Trusted Automated eXchange of Indicator Information) which defines the services and message exchanges which allow sharing of actionable cyber threat information across organisational boundaries.

We are finally starting to see some standardisation which will ensure that security teams are all talking the same language. On top of this we have to be able to ingest this data, validate its use and implement it into our security infrastructure, easy right?

The biggest issue for SOC's in relation to Threat Intelligence is understanding what is "Actionable Intelligence"? For most organizations this will have a cascading degree of severity depending on its source and age. The table below shows examples of typical Intelligence and their level of trust within a SOC:

Source Trust Notes
Internal Analysis (Packet Analysis, Hashes, File attributes) Exceptional Intelligence gathered from direct attacks against the company/organisation will always have the highest value as it relates to confirmed attacks.
Government Agencies High Typically targeted for specific industry verticals and peer groups. Entirely reliant upon an external agency to identify threats. No ability for a company to have the threat background to the individual intelligence artifacts.
Industry Peers Medium - High Intelligence may however only be suitable for one company in the case of targeted attacks.
Vender Blacklists Low Taken from a Vendors exposure to the threat. Will be generic in focus. Good deal of False positive work required on alerts
Open Source Blacklists Low Considered "Low Hanging Fruit"
Good deal of False positive work required on alerts. Threat Actors will change their TTP's upon being aware of having been identified.

One of the biggest issues with Intelligence is knowing the Intel's age to be able to give a weight to its usefulness. This is particularly of importance when assessing Beacon Domains and identifying how long they have been in operation. When it comes to the malware itself the usefulness of some data has been drastically reduced. In the age of polymorphic malware 99% of unique MD5 hashes are being seen for 58 seconds or less according to the Verizon DBIR 2016.

Managing all of this information is obviously a daunting proposition so how do we take all of this information and make it useful in our organisations? Dedicated Threat Intelligence (TI) Analysts are required. They must first understand the Business and what its Risks and potential Threat Actors are before going about establish a Threat Intelligence management program within their organisation.

Business and Risk

  1. What are the risks to the business (confidentiality, availability or integrity)?
  2. What are the overall cybersecurity risks in the industry sector? High, reasonable or low?
  3. What Assets are of value to threat actors?
  4. What legal or regulatory requirements exist related to the information to protect, either stored or in transit?


  1. Who are the threat actors targeting the business?
  2. What does the threat actors want?
  3. How will Threat Actors plan to get it?
  4. What data does the SOC require? Can we act on it (manageable, automated)?
  5. What is the effectiveness of external and internal threat intelligence?

The Diamond Model takes this a stage further and allows Threat Intelligence Teams to utilise a standard approach to classify intelligence and attacks.

Tools such as CRITS, MISP or one of the many other Open Source or Vendor Cyber Threat Intelligence management tools enable the TI Analyst to create campaigns and track specific adversary TTP's. They are able to weight the value of the Intelligence and "retire" out of date Intelligence which has no further business impact and which can cause false positives or worse yet cause performance issues with the detection equipment. These tools are able to integrate with STIX and TAXI to ensure that information can be added easily without the analyst having to manually create all of the information. They can also be used to submit samples for analysis to various security tools i.e. Sandboxing (Cuckoo, FireEye), hashing databases, and Virus repositories amongst others. This gives the TI Analyst a greater degree of confidence in the Intelligence at hand and gives them a better Threat Profile and allows them to choose specific subsets of data to output from the Threat Intelligence database into the organisations detection capability.

The TI Analyst rapidly becomes a key persona within any SOC when they work closely with the Content Engineer and the Analyst team to review and update the SOC's detection capability based upon the key Business Security Threats. The SOC Analysts notify both the Content Engineer and the TI Analyst when a new threat is detected from Packet and Log analysis. In turn the Content Engineer is able to create new detection capabilities based on the Attack Scenario and Use Case whilst the TI Analyst assesses the information and correlate's the data against known intelligence and campaigns giving attribution where possible to the attack. The information from the SOC Analysts is also used by the TI Analyst to expand the SOC's Threat Intelligence knowledge base of current attacks.

Armed with sufficient knowledge about their adversaries the TI Analyst is able to search the Darkweb using fictitious personas and unattributable internet connections to take the Threat Intelligence program from a purely reactive stance to identifying when Future attacks will be instigated by monitoring attacker forums, newsgroups, twitter feeds and other communications channels.

Threat Intelligence is therefore a core requirement for any Business Security program. With a sufficiently skilled TI analyst who is aware of the threats to the Business and the adversaries facing them they can easily increase the detection rate for the SOC allowing them to detect attacks earlier on the Cyber Kill Chain TM

For more information on Threat Intelligence please see Demetrio Milea's blog "Measure Your Readiness - Threat Intelligence Program"