Pirates (on the High Seas), Graffiti and Big Data

Big Data is starting to have significant and wide-reaching impact on our lives.  And, it can be found in quite unexpected places.  +Nate Silver's prediction of 2012 US Election has been making headlines on the web for the last three months.  The accuracy is surprising, using Big Data to achieve it, probably not so much.  However, following are two scenarios where Big Data’s value is not so obvious.  The first one shows how Big Data analytics has been used and the second one demonstrates the potential of Big Data for dealing with minor crime and benefitting communities.

Pirates (on the high seas)

Although we’re now in the 21 century with advanced technology and military hardware at our disposal, piracy still remains a significant problem.  Oceans Beyond Piracy, a US-based consultancy, estimated that losses from piracy reached $7Bn in 2011. 

However, in 2012 the International Maritime Bureau reported a 54% drop in piracy incidents in the first half of 2012 compared to the same period the year before.  The reason for this drop was the fact that law enforcement agencies turned to Big Data to try and prevent piracy.  The amount of data available to Government agencies is analysed in order to predict where the incidents may occur in the future.  The information is then plotted on a map using advanced mapping software by Esri and used by the relevant maritime and Government authorities.  

The results, as we can see, are simply staggering. 

… And, graffiti (in the city)

The above example brings us to another potential use of Big Data - reducing graffiti appearances in our cities.  According to a report from the Australian Institute of Criminology, graffiti costs around $1.5 billion a year in Australia, based on conservative estimates.  Graffiti can be classified in several groups with the offenders in each group possessing a number of distinct characteristics.  The most prevalent type of graffiti writer, accounting for over 50% of all people engaged with graffiti, is the tagger.  The report also lists some specific traits of graffiti writers: 

• tagging is more common among teenagers and piecing, or graffiti on murals, prevalent in the group of those 15 years old or over;
• likely to be alienated from school in some form;
• majority have strict self-imposed rules where they will graffiti;
• majority are males between 12 and 25, etc.

Thus, if we look at how to deal most effectively with graffiti, it is clear that focusing on taggers will provide the “best bang for the buck”.  They also are the easiest group to deal with in terms of predicting the location their future actions since their mobility is limited, because of age.  Thus, using data from schools, reports of graffiti incidents to police and local authorities and data from cleaning operations it is possible to build a predictive model in relation to graffiti.

Similar approach has already been applied by the Memphis Police Department in relation to various crimes with great success.  The Department is using analytics to predict incidents.  The report from their analytical solution may say “You’ve had three robberies that occurred between 10 p.m. and 1 a.m. on Monday”.  This allows patrols to be deployed in areas where they will have the biggest impact.  

In essence, this approach is similar to the one, mentioned above, and used to fight pirates on the high seas. And, in relation to graffiti it can be equally effective because of the similarities in available data and the characteristics of the crime.

Can Big Data Crack Fraud?

How many jailed criminals receive unemployment benefits? How many people are working while claiming unemployment benefits?  How many unemployment benefits recipients are driving high-end BMWs? These questions are not always very easy to answer.

Part of the problem is the increasing number of services delivered through online methods.  Registering with Medicare or Social Security and then receiving benefits can be done in a fully virtual environment.  The situation is complicated further by various jurisdictional boundaries - local, state and federal governments.  This problem is probably less pronounced in countries with two levels of government, but it still exists, nevertheless.

However, the above questions are becoming easier for authorities to answer by using Big Data. And, smaller jurisdictions, with correspondingly smaller budgets and "narrower" view of their constituents, are going to be some of the biggest beneficiaries.

An example of using Big Data to identify fraud is the work done by +LexisNexis in US.  The company compared Medicaid recipients against vehicle registrations.  The check discovered quite a number of people receiving Medicaid benefits and driving everything from Bentleys to Aston-Martins to high-end Mercedes-Benz vehicles.

According to Andy Bucholz, director of Government Fraud Solutions for +LexisNexis, a much more holistic view of citizens and their activities is very likely to solve some of the fraud-related problems.

Read the full article in +Government Technology by Wayne Hanson here.

How Big Data Is Changing the Whole Equation for Business

Businesses in a range of industruies are finding new ways to use the data available to them. Quite often business as usual gets in the way of any Big Data initiatives. After all, humans (and organisations) are inherently resistant to change.

Here are some examples of organisations using data to transform their businesses.

Human resources - Caesars Entertainment is analysing health insurance claims of its employees. Since 2009 the company has saved $4.5 million.

Product Development - Zynga, the game maker behind Farmville, captures 25 terabytes/day and uses that data for customer service, quality assurance and new feature research. For example, in FarmVille 2, animals were made more central because, surprisingly for the company, players interacted more with them than originally expected.

Operations - UPS combined GPS information and data from fuel efficiency sensonrs to reduce fuel consumption by 8.4 million gallons and cut 85 million miles off its routes.

Read the full article in +The Wall Street Journal  by Steven Rosenbush and Michael Totty here.

Big Data and Business Intuition Work Together

The question of gut vs. data is becoming more and more relevant considering the terabytes of data available in each organisation. However, there's is a happy medium according to reports from the Economist and +Harvard Business Review. The trick is how to combine the two.

According to management expert Dr. Chongqi Wu gut-based and fact-based decision making complement each other and shouldn't be seem as competing approaches. According to Dr. Wu "Data just makes executives better gamblers".

There are times, of course when one of the approaches is better than the other. For example, it's much better to rely on data in operational decisions that can be automated. On the other hand, when dealing with incomplete information, setting strategy is one example, decisions based on intuition are can be the better.

It's important to differentiate between the initial hunch, which is quite often correct, and the subsequent assumptions that have almost no intuitive value.

And, the old adage of "garbage in, garbage out" holds true in this instance, as well. We need to learn to ask the right questions of the available data.

Read the full article on +GoodData by +Cliff Cate here.

Big Data Questions for the Layman

• 90% of data was created in the last two years. Why is that? 
• How does that affect security professionals? 
• How does that affect the everyman?

According to various sources 90% of the data in the world was created in the last two years.  The main reasons are: the rising popularity of social networks, our willingness to share more and the type of information we create/consume/share.  Let’s have a look at each of these in turn.

Rising popularity of social networks

Facebook, for example, grew from 197 million to 901 million active users within the space of 3 years (source: jeffbullas.com).  Google+ figures tell a similar story, within a year its user base grew from less than 50 million to 500 million (source: farotech.com).  These are the two behemoths of social networking; however the figures from other networks, such as LinkedIn, Twitter, etc. suggest a similar story.

Willingness to share

And, this growth has made all of us much more comfortable using the networks, connecting with family and friends, posting photos, videos, songs, etc.  

In a sense, this is similar to the e-commerce trends. In the early 2000s only a small number of people were transacting over the Internet and the security, privacy and other concerns were quite high, and justified.  These days hardly anybody pays attention how the transaction occurs.  What is important is the result – a purchase made at an acceptable price with almost 100% guarantee of delivery.

Mark Zuckerberg even speculated that the amount of information we share doubles every two years.  

Type of information created/consumed/shared

Until recently creating and posting “rich media” for others to see and use required a certain amount of technical skills.  At present, with a large percentage of population (50.4% in US, according to Nielsen) carrying smartphones in their pockets/handbags, it’s never been easier to create a 3-min video of little Johnny playing with his sister. And, the size of this video on iPhone 4 would be 240Mb, while on iPhone 4S or 5 that figure would go to 540MB. 

Even with compression, let’s say when uploading to Facebook,  the data amounts are staggering compared to a spreadsheet considered “state of the art” only a few years ago (2005 seems somewhat pre-historic these days in “Internet chronology”). 

This type of data explosion is certain to affect a growing number of professions.  And information governance and security professionals, will be amongst to first that will have to adapt and deal with this new reality.  Organisations are already starting to recognise the importance of Big Data and how it can deliver competitive advantages.  As a consequence, this realisation will further accelarate the drive to start using these technologies.  

Thus, the question in the mind of many executives will become “how to do it”.  It’s certain, existing systems and infrastructure are not very well suited for Big Data purposes.  Some of the more obvious options would be:

• develop expertise in-house and introduce new infrastructure;
• outsource Big Data processing to companies specialising in this area;
• a mixture of the above;
• or a new model, e.g. utilising Platform-As-A-Service (PaaS), Software-As-A-Service (SaaS) or something currently being dreamed/prototyped in a garage somewhere in the world

Any of these scenarios imply a certain amount of change.  And, how this change is decided upon, implemented and then managed will be of concern to governance and security professionals.  Why? They will need to provide advice, help other units within their organisation and monitor the new processes afterwards.

This is a tide which is already irreversible.  Organisations can’t afford not to deal with Big Data because their external stakeholders have already embraced it.  Our lives have been enriched by new technologies and we will continue to share lots of photos and videos.  We will ask for sensors to be attached to us so that we can monitor our blood pressure, or how well we exercise or sleep.  And, because we know that, through various sensors, a trained professional can be monitoring our parents’ health, living on their own, 24 hours. 

Big Data has made big promises and now the general population is keen on those being delivered.