Machine-generated data analysis and visualisation tool Splunk has shifted from being a pure IT operations tool at New Zealand-based classifieds site Trade Me to a way of delivering near real-time insights to the business.
"Like lots of people we started using it for IT ops and as a syslog replacement," Trade Me head of infrastructure Matt van Deventer said. "We would have all of our Windows event logs and all of our Linux logs going in there."
The company participated in the Splunk 6 beta earlier this year and upgraded when the latest version of the application hit general availability at the start of last month.
"We upgraded our whole infrastructure probably within a week of it going GA; we jumped on pretty quickly because we really like it," van Deventer said.
"They've done a lot of work in the visualisation engine [for Splunk 6], so we can now create data models for our data – my guys can spend a little bit of time correlating some data and creating a bit of a structure and then we can just point business guys at it and they can drag and drop and create pivot charts and pivot tables."
"I've got a handful of Splunk ninjas on my team because we've been using it for the longest, and they end up fielding everyone's questions. So being able to push some of that stuff back out into the business so that they can help themselves is really the biggest thing for us," van Deventer said.
Trade Me has been using Splunk for several years now. Originally, the company purchased the smallest licence available, but Trade Me has "had a few sort of 'a ha' moments over the years," van Deventer said.
"The canonical example is almost exactly a year ago: In November last year we were having a bunch of issues with our image serving."
Trade Me has around 600 million images on the site, with another seven images upload every second, and about 10,000 image requests per second.
"There's a lot of stuff going on, and when something goes wrong it's very difficult to know where about in the stack the problem is – whether it's in the caching layer or the network layer or the storage layer or somewhere in between," van Deventer said.
"Being able to throw all the logs into Splunk enabled us to build a couple of dashboards that went up on the wall, and everyone could see when the problems were happening and it enabled us to find the root cause and fix it."
That became the business case for the next licence upgrade. All of Trade Me's staff now have access to Splunk, van Deventer added.
"Over the last year we've increased our licence and we're now using the Splunk DB Connect app to talk to some of our production databases. We can get our data out of those databases in near real time, and that gives the business the ability to write searches and see what's happening when we put new features out on the site."
The company is putting around 100 gigs of weblogs into Splunk every day, as well as image logs and business data.
"We've only very recently upgraded to Splunk 6 and we've got a lot of work to do there," van Deventer said. "Once we've got that all bedded in and the business happy, then that's when we'll look at adding more data in there."