GreyNoise | How we built IP Similarity

We briefly introduced IP Similarity previously, but now we want to dive deep and show how we made this idea a reality. The first goal of IP Similarity is to encode a GreyNoise record as a numerical feature vector. This is just an array of numbers that somehow represent all of the data we have in a GreyNoise record. This representation is extremely useful for machine learning and any numerical analysis. From this point we can quantitatively measure how far away two records are, cluster groups of records together, and build all sorts of classifiers. This is the ground floor basis for applying machine learning to GreyNoise data. But, getting there is hard. Our records contain a vast amount of unstructured and semi-structured textual data. User-Agents can be nearly anything you want, from ​ to ​ Web paths can be as simple as ​ or complicated like ​ Ports can be any or all of the 65,535 available values. The list goes on. In order to turn this complex multi-modal data into a fixed size nume

We briefly introduced IP Similarity previously, but now we want to dive deep and show how we made this idea a reality. The first goal of IP Similarity is to encode a GreyNoise record as a numerical feature vector. This is just an array of numbers that somehow represent all of the data we have in a GreyNoise record. This representation is extremely useful for machine learning and any numerical analysis. From this point we can quantitatively measure how far away two records are, cluster groups of records together, and build all sorts of classifiers. This is the ground floor basis for applying ma