SIG 2008

Sorry about the leave of absence from blog duty - been traveling a lot. Long days and lots of work don't go well with writing text in the evenings. Also, there's been a lack of any seriously interesting news (besides for the Comcast bandwidth cap, but that's been debated to hell and back anyhow)

One of the benefits of travel, however, is that you get to go to cool places. One of these cool places would be the Security in Government 2008 trade show in Canberra, Dropbearland. If you're into things like next-to-unbreakable boxes, very mean looking electric fences, seven sorts of barbed wire, access control systems and things like that, it's a pretty nifty place. Definite geek appeal - but not very IT.

They had two things that were both IT and nifty though, and they cover the same general idea from different angles.

nuix
nuix is a company specializing in what could best be described as data mining email on a massive scale. I got a quick, yet fairly thorough rundown of how their stuff works. Extremely cool visualization makes it pretty easy to follow the path(s) a mail has taken through an organization. Combine this with e-mail retention policies (or even just storing a copy once it passes the server) and you've got a pretty capable tool. They have a video on their site that gives you the highlights (sure, it's a bit sales-heavy, but it's on their website, so I can't blame them really..)

Note the bit where the guy says "we have organisations who actually want want to plug us into the internet network going in and out of that country" - granted, we're likely talking about a small'ish country here - but it gives you a bit of a scale to things.

All in all cool stuff. If I were a CIO or anything with more users than I knew the face of, I'd definitely opt for something like this.

endace
These guys make some pretty hardware and the appliance with the worst name ever, NinjaProbe (geek points, but seriously..) - this isn't news, they've been going at it for a while. What's news for today though, is their new analysis center software. In short: Their appliances are capable of storing a metric shitload of network data to disk. Their center software is capable of mining these metric shitloads for useful data and extract only what you're actually interested in for further processing. It's a niche, but it's one that makes a lot of sense to fill.

I didn't see a demo of this, so no idea how well it works, but all in all they too seem to have their shit together.

The punchline
The common denominator here is of course data mining. In both cases we're seeing massice data mining simplified to a level where pretty much anyone can be trained to use it effectively and it's - comparatively speaking - extremely affordable. What does this mean then?

1. Your local intelligence agency might not be able to code stuff like this themselves, but they sure got a lot of money to throw at the problem. Assume that they have access to these tools for better or worse (I rather like the thought myself, but I have some faith in law enforcement)

2. These are tools that are within the budgetary reaches of your local university as well. I don't find it at all a stretch to see that some of them would want to 'ensure the safety of their students'. Which might well mean that mailing nasty horrble things such as texts containing the word 'fuck' to other students would be seen as an offense. Or something equally silly. (I don't have as much faith in academia. Techies tend to be allright, but they're not the ones who are calling the shots when it comes to that)

3. This is just scraping the surface. I'm sure we'll see quite some more interesting uses for data mining internet services and comms soon enough.