A story of data and perseverance

Let me tell you a story about data. Why? 

Well, I (and many others) believe that story telling is important in business as it gives a real-life insight into how things come about, uses less business jargon, and can bring to life a specific journey without the need for a presentation or anything other than a story. Of course, the story must be true.

A long, long time ago…

This story begins 4 years ago late in the evening in a hotel bar in Aberdeen. I was with my rugby club’s mini’s youth tour. I was a coach and a so-called responsible adult. Hmmm. Our kids were running about causing havoc in the hotel when some of us parents were sipping an orange juice.

There were two other parents that I had not met before and I got talking to, and as luck would have it, we had hit upon the specific topic of data after we got to that moment when someone says: ‘What do you do?’. 

‘I have worked in IT and Systems for the last 3 decades’ I told them, and now own a business delivering consultancy in the social housing sector.

They both owned a data science company that had grown considerably and was delivering great projects for a wide range of companies worldwide, all in the private sector of retail, banking etc. 

Some great customers there I said and jokingly (those who know me, will know I can just blurt things out in the moment) asked if they have ever done work on data in the public or third sector. 

They hadn’t, but there in lay the challenge.

‘Why not?’ I asked.

Our first steps in #ukhousing

This was four years ago, give or take a few months, so those reading this from a social housing track will know that although some were looking at data analysis, reporting (yuk), even a dashboard or two, it wasn’t by any means top of the social landlord’s agenda. 

‘It never hit our radar’ they said, and ‘nobody has ever asked from that sector’ they supplemented. 

I know I used to be a lot more critical back then. I was after all still a young pup in my forties, albeit my late forties, but I did say that one of the issues with the social housing sector, and still is to a degree, is that they are very insular and don’t look for solutions or best practice from the outside. 

So, was there a way we could join forces to look at data solutions for the sector by using my consultancies knowledge of the sector and their specific expertise in data science and engineering?

You bet there was. 

We designed some services that brought data analytics and Power BI dashboards to several landlords, and it worked well. However, in business some things throw you off-piste. In this case, they were acquired by a large American company and were taken in some different directions that meant our dream of saving the social housing worlds data-based problems through data science, engineering and problem solving was off the table.

And that was that.

Let’s try again!

Until that is the founders of that business, the two I bumped into that night recently brought up the subject of trying again. They now had a new business full of talented data scientists, engineers, analysts, and problem solvers.

And so, we did try again. This time it’s even better, and it’s now a service that is about to make some serious in roads to helping landlords with their data-based problems. 

We both love that joining of forces where we can bring some of the UK’s leading minds in data, working alongside my own company that is so determined to make a difference and create change in the sector, this time with one of your most valuable assets, and you could argue, perhaps over that late night orange juice, your most important asset. Data!

Find out more and get in touch about our brand new Data Improvement Services.

Not all swamps are as cute as Shrek’s.

You may have heard about a Data Lake, but you might not have heard about a Data Swamp. Yes, they are a thing.

We all know that more and more data is being collected by companies. The need to collect increasing amounts of data and of course store it, risks creating data swamps. The word swamp came about as a way of describing the out of control, somewhat messy version of what was intended to be a data lake. I generally use the term for any scenario where the overall data repositories are messy, somewhat disorganised.

A data swamp is a horrible place, unlike a data lake which is a beautiful place that allows companies to retrieve and use their data efficiently and effectively.

The uglier, smelly, dangerous even, data swamps can make both those goals very difficult and perhaps impossible. They are a sign of mishandling data for sometimes years. They are a sign of poor data management and strategies.

However, all is not lost.

There are a number of strategies you can employ to ensure you don’t sink into the murky depths of a data swamp, which let’s face it, a number of those reading this will have one foot in already.

Data management strategy and governance

Ask yourself this. Do we have an actual over-arching data strategy? In response to this we have heard the following from landlords. Surely, we don’t need one if our systems are working ok. Surely that is for really large companies? Surely all we need to do is follow GDPR. All wrong, and indeed dangerous.

Excellent data governance is what equips your organisation to maintain a high level of data quality throughout the entire data lifecycle from creation to destruction.

Data governance defines how to work with data, who should access it, handle it, how long you retain the information for, deciding where the data is stored and so much more.

It simply isn’t enough to assume that these clever systems you have spent a lot of money on will do it for you. Sure, setting up roles and permissions in a system, in your active directory, and perhaps even making these the same thing isn’t enough. You need a strategy and indeed associated policies. You also need a strong governance model. They all go hand in hand so don’t scrimp. Get started, or get improving now, before it is too late.

Set yourself some homework to avoid a data swamp

You don’t get results from sitting back and doing nothing. A good strategy and policies ensure you have regular homework to do. If homework is a term that fills you with fear from your school days, then it’s easy, think of it as admin or maintenance. Maintenance is very apt as we all know that working parts need maintenance for them to continue to work at their best. Data is no different. It can very much be seen as a moving part in the operations of your company.

Tasks you would expect to do need to be formalised, structured and set owners, and those responsible for carrying them out. This is absolutely critical to the ongoing success of data management and turning your data into gold. One specific task that I wanted to highlight is that of cleaning your data. It links to the later area of data quality. If you don’t ensure your data is a clean or indeed tidy (not messy), then a great deal else falls into place neatly. Analytics make more sense, report and dashboards can be trusted and staff and customers are happier.

Reach for quality: having irrelevant data, old data, nonsensical data is simply not acceptable.

This is the cardinal sin of a great deal of companies across the world, in many sectors. It’s not something specific to social housing. We talk about rubbish in-rubbish-out. Well, I certainly mention this regularly. Imagine you need to make a decision based on the life span of a boiler part, but this information is messy, with some wrong dates in there. It may be that this was human error, or perhaps even the way your new system managed the data. The point here is that if you are not on it, and the data is not accurate, relevant or indeed even spelled wrong then decisions based on this data can end in disaster, literally.

The main take away here is that there is homework to be done. There is maintenance required, and if you don’t do it, and don’t set it out in the strategies and policies then expect a data swamp, expect mistakes, expect costs to rise and expect unhappy customers.

The next steps towards preventing a data swamp

So, above outlines three straight forward yet challenging areas to get sorted. To be fair, you can apply these to not just avoiding your data lake becoming a swamp, but you can simply use them as guides to better manage your data, lake or not. Need some help with that? Get in touch!