Sun shining, birds twittering, colleagues talking about heading out for lunch.

Sun shining, birds twittering, colleagues talking about heading out for lunch. And I’m stuck at my desk wrestling with pivot-tables and massive datasets.

No twittering for me, just twiddling of thumbs while I wait for Excel to churn through the refresh of a graph after I updated a data dimension.

I’m doing this to analyze a section of a customer’s batch schedule, usually a month or so, taking the contents of large log tables, rapidly process it through our Diffuser framework and interpreting the results.

I think, “There must be a better way. Surely someone has come up with a smarter, faster way to process large volumes of data which doesn’t require a masters degree in Excel and an egg-timer.”

Ultimately all this data crunching is extracted, cut, pasted, trimmed and cajoled into a presentation to show all those hidden pain points of a SAP system’s inner workings.

The end results are striking. However what they are not is elegant, interactive, portable or adaptable.

Think of all those phrases in this area:

• Agile visualization
• Trusted data discovery
• Simplified analysis

Nope, Excel is none of those!

Let me be clear, I’m not talking Big Data here.

That is a subtle but important distinction. Yes, I’m dealing with large amounts of data, but not massive – only about half a million distinct rows.

I’m not looking to find any non-traditional methods of processing either. I like to think of it as Small Data; analyzing the minutiae to tease out some meaning.

The age of IT providing a static report and the end user consuming it are gone. A business has to react faster to changes and know more about the business and the best businesses have done it by turning this old model on its head.

I remembered I wrote about almost this exact subject in my graduate degree dissertation, addressing the conflicting aspects of simple presentation and complex data models.

That was many years ago but what was relevant then about representing large, seemingly disparate data and turning it into information is still relevant now – and given the massive rise in volume and scope of data being collected, you could argue it is critical now to avoid drowning in the mire.

Let me pose two examples and two questions:

• One table of data
• One graph of the same data.

Which one tells you more?

Which tells you something useful?

It takes a certain mind to see meaning in a large table of figures, text and numerics.

But the human brain is exceptional at pattern recognition. Show it a Where’s Wally picture and despite your frustration it is remarkably good at picking the stripy little fellow out.

Show your brain a graph, table or well-crafted representation and it can spot a trend or anomaly almost instantly.

I came across a great examples representing time-based datasets and there is a lot of data going into them but if was presented in a traditional way would not be anyway near as easy to interpret.

Here’s another example of visualizing source code repositories using an interactive tree.

Now we’re getting somewhere.. elegant, useful, informative.

My goal is to use this to help someone see for themselves an engaging and persuasive story which they can adapt themselves, gain insight and possibly with the help of Basis Technologies get faster, better SAP.

“The customer is king” is an adage which I think now applies to reporting and data too.

By providing these models and a frameworks, the data is transformed into information. It now has value, you can manipulate according to your personal requirements at that exact moment. If you want an overview then it shows you trends but if you spotted Wally, that interesting blip, and want to know more then it should give you the ability to drill-down, mine your data and resolve out the detail.

I want this to translate into a more malleable report framework – less of my time with Excel’s formulae and pivot-tables and more time applying my knowledge.

Well that’s the vision. How on earth am I going to get there?

So I do what everyone does when stumped – turn to and start typing. “ visualisation tools”, “visualise SAP batch analysis”, “SAP analytics”.

After realising that I had to start using the US spelling, my searches turned up a trump that had somehow been under my radar for years.

I found something which seemed to tick all the boxes of what I wanted. SAP data sources, modern powerful UI for design, trusted data manipulation and open-source visualization extensions and ability to create infographic-style stories.

Let the journey into the world of SAP’s data visualizations commence – with a bit of hard work I think we’ll get something onto your desktop which will help your business.