What is big data?

What exactly is big data, is this just the next “BIG” buzz word, will it follow the same path as Y2K or is it here to stay?

Big data is exactly that what it says, it is large amounts of stored raw data.  Studies show that the data generated every two days equals the total amount of data that was created up to the year 2003 and over 90% of all the data in the world was generated over the past two years.  So where does all this data go, who has it and what are they doing with it?  Here is a quick guide to how we generate so much data:

Every minute:

  1. We send 204 million emails
  2. Generate 1.8 Facebook likes
  3. Send 278K Tweets
  4. Upload 200K photos to Facebook

And this is one a drop in what is the big data ocean.

So with all this information floating around how is it being used?

Big data is being used every day across all industries from retail to farming to health and within government bodies.

In industry the use of big data helps business understand and target customers.  Yes those little fobs your local Tesco, SuperValue and Petrol station give you are tracking everything you do within their store.  Was it by coincidence that you received the coupon for your favourite shampoo?

With Big Data improving our shopping experiences, improving health systems and the fighting  against cancers and other serious illnesses it has now turned its attention to the fight against crime and joined the Los Angeles Police Department, and no it does not come in the form of Tom Cruise or Colin Farrell.  After all police departments around the world have their customers too and they need targeting.


Professor Jeff Brantingham and a team at UCLA studied over 13 million recorded crimes, spanning 80 years and applied an algorithm that is used to predict the likelihood of aftershocks from earthquakes.  The original algorithm that looks at the probability that aftershocks occur close by in space and time, was developed by Assistant Professor George Moher.  Applying a theory that aftershocks happen in close proximity the teams approach was the same when looking at crime and human behaviour.  Using the data they wanted to see if there was any relation.  Strangely enough the patterns were similar.  Although it couldn’t prevent any of these crimes as they had already happened, they decided to build on the algorithm and use live data to see if it could predict potential crime hot spots.  With some tweaking and the joining of forces with the company PredPol, they can now predict where crime is likely to happen on a given day.

The software breaks the patrols  in to 12 hour shifts covering a 500sq feet geographical area, where it has predicted that criminal activity might occur.  As the patrol officers had a new commanding officer it took time for them to warm to their new partner, as who knows crime best a computer or a police officer?  But attitudes changed as the city has seen a 33-percent drop in burglaries, 21-percent drop in violent crimes and a 12-percent decrease in crimes against property.  On Thursday, 13 February, 2014, LAPD’s Foothill area recorded zero crime activity over a 24 hour period, “a day without (recorded) crime” the first in their fifty year history.

So Big Data is making the world a healthier, safer and better place to shop.

BBC documentary on LAPD Big Data.

Leave a Reply

Your email address will not be published. Required fields are marked *