Our Blog

where we write about the things we love



A gateway to big data: geospatial business intelligence

A Gateway to Big Data: Geospatial Business IntelligenceWith the continuing hype around the want, need, or even the “me too” aspects of being able to join the ‘Big Data’ movement, it is only a matter of time before BI will, by default, incorporate this nebulous concept.

I use the term nebulous because the notion of ‘Big Data’ can depend on which part of the organisation or individual person you are talking to, or perhaps even the industry they work in. So far I don’t think I have heard of a single right or wrong definition for ‘Big Data’ but it normally falls into two camps: huge, constantly updated datasets; or the capability to look at multiple complicated datasets that, at face value, have no relation to or bearing on each other whatsoever. Maybe it’s both at the same time. This does, nonetheless, boil down to three classifications: volume, velocity (the rate of change of the data) and variability (the multitude of different formats and complex datasets available).

Another common scenario in BI is that most businesses, and certainly the majority of New Zealand businesses, currently struggle with the effective use of ‘Small Data’. “How can my company unlock the most insight and potential from the structured organisational data we keep within our own four walls?” Unlocking that insight and potential is what we BI folks do for a living.

If ‘Small Data’ is a concept we are continually educating our clients about, and providing valuable solutions to, imagine trying to promote a concept of how to access and use multiple datasets that provide no apparent direct bearing on their business, or that come from different market segments and industries, but yet allow them to obtain better insights into their organisation. This generally appears irrelevant, too costly or too complex and gets filed in the ‘too hard at the minute’ or ‘not for us’ baskets and could be a reason why, despite the current buzz and attraction of big data, there is a slow uptake in the New Zealand and Australian marketplaces when compared to the likes of the US.

But does ‘Big Data’ have to be this difficult, or does it have to mean that you must have an understanding of all the new and emerging technologies like Hadoop, and cloud processing? Although this understanding is certainly valuable and will become a standard requirement of BI in the future, I don’t think you need to have this right now to be part of the ‘Big Data’ movement.


So how does geospatial provide an accessible alternative?

Looking at geospatial technology, it’s been around for the best part of 30 years and is robust, proven and becoming a mainstream technology; and for more than just mapping. With the advances in databases such as SQL Server being able to handle and query spatial information, geospatial provides a gateway for organisations to easily look outside their walls to harvest and use beneficial data from multiple market segments and sources. These will more often than not have no common bearing on each other but can be leveraged by organisations to better their insights into their own business. Combining traditional BI with this technology can greatly enhance organisational knowledge and is an easy first step into ‘Big Data’.


How would this work?

First, abandon the notion that geospatial means we’re going to start producing digital maps (although this can be a powerful way to present BI, and Microsoft is making good headway with the addition of plug-ins like Power View and Geoflow), or that I’m suggesting we become a GIS company with specialist tools and software…..I’m not. I am suggesting that we consider focusing purely on enhancing the business oriented outcomes GIS can provide traditional BI.

As BI practitioners, we can go into an organisation and help them structure data from single or multiple systems, organise it, pull it into a data warehouse, build analysis cubes, present reports and dashboards and allow them to gain new insights, increase productivity, save time and provide a more comprehensive view of their business.

This is great; it gives a comprehensive structured view of their company from a number of aspects such as sales, or inventory management, or receivables. An organisation can identify trends, or anomalies, and understand which levers to pull to change the results they are seeing. But what of the factors outside the business? How do these influence their results? This is where geospatial plays a part.

A very large proportion of data contains some element of geographic data. If you have customers, you have geographic data; a retail network, you have geographic data; logistical costs, you have geographic data; a distributed workforce…well you get the picture. It’s the position on the globe the data provides that becomes the common attribute. No primary or foreign keys in your databases, no relational structures, no common entities – the point on the globe now allows us to use spatial analysis to enhance our insight into business data. So what does this mean for your average New Zealand business, or your average sized business?

We can start to take any information that is attributed to a location, an area, a route, from any industry, and enrich our business data analysis and insight with it. Because we know where it is, rather than what it is, the data can be from any provider, be in any structure, and – providing we can read it – in any format. The data doesn’t, in any way, have to share identifiers, or be presented at the same granularity, as we can use spatial traits to correct for this. It is this fundamental idea that very few business are realising and leveraging to their full potential. ANZ bank for example now use traces from heavy goods vehicle movements and volumes of these vehicles across the country to predict national GDP with a six-month lead time with surprising accuracy.

The other major benefit in this area is that more and more information is being made available spatially, and more and more of this information is being made freely available; so if it’s there, it costs us little or nothing, and we can use it to answer our business questions, why aren’t we taking advantage of it? Another benefit is that more commonly feeds are being made available in real-time, from mobile devices or organisations. The value of understanding your customer’s opinion as they leave your store and Tweet or Facebook about it can allow you to more rapidly adjust your approach to sales or marketing. Spatially this can be analysed on a store-by-store basis.


The possible business benefit

Why wouldn’t we want to be able to say to Department Store X: “Your business information shows X, Y and Z trends in each store by product type and you now have the ability to obtain a comprehensive picture from multiple business systems in a single location. This provides you a more effective platform to monitor and control your business”. What a great story.

But imagine if we could extend this: “We now know where your most profitable stores are for the sale of electronics, and we know why electronics are your best seller in these areas, because we know where that spend originates. From that spend origin, we can profile the population creating that spend, identifying your ‘ideal’ customers based on socio-demographic data. We also know where you can find more of those ‘ideal’ customers that you don’t already sell to and how far they are prepared to travel to your store to shop. So should you think about relocating or opening a new store? What percentage of total market share for that product category do you hold in any one area? We also know from your recent advertising campaign responses which forms of adverts were most effective against each population segment, and this allows you more control over your marketing budget by effectively focusing investment in specific types of targeted advertising campaigns…”

Not only does this improve the business insight for the team with whom we engaged initially, but we are also very easily able to communicate a beneficial message across the wider organisation (in the case above marketing and the retail network planner), with minimal cost and effort, and the ability to re-use the components we have already implemented. Suddenly we are talking about enterprise-wide BI as well.

The options and additional insight that can start to be leveraged or obtained are really only limited by the data available and the creative ways in which the information can be applied to the business data you already have.

I think on that basis and the fact this technology has been around for such a long time, it provides businesses an easy way to adopt proven methods for integrating a vast variety of disjointed, complex, and frequently changing data. Perhaps “Big Data” is a little more accessible for the smaller organisation than we first thought?

Posted by: Russell Bowden, Business Intelligence Senior Consultant | 13 May 2013

Tags: Business Intelligence, Microsoft, SQL Server, Intergen, BI, Big Data, Geospatial

Blog archive

Stay up to date with all insights from the Intergen blog