How to proceed in the age of big data?

A couple of weeks ago I read an article in the New York Times about the age of big data, and today at a science and technology conference I got into a conversation about the same thing with a US public health official.

Much has been written (and I am a guilty party) about Google’s quest for information, including allegations of infringements of privacy etc, but not all of this capability should be seen in a negative light. I would like to give you a few examples of why.

A wealth of data

Google collect all of the search terms used by every user and categorize them. Let’s take a hypothetical situation. You are director of a large hospital inManchester. What can Google tell you about your job? Well probably a lot, let’s say that this week there is an enormous peak in the search terms “Flu symptoms” used across the Greater Manchester area, or “rash on back and neck”. Indirectly the knowledge of these search trends tells you that you should prepare your hospital, because late next week you will have a massive influx of patients with the Flu or some other contagious disease as it takes hold of the population.

This information is potentially lifesaving, as one of the main problems with epidemics is they come out of nowhere and so health centres are not properly prepared.

Search terms can also give an indication of how the housing market will behave too, with a rise in searches for houses in a certain area being reflected 6 months later in new sales. The type of house searched could also improve planning, as developers would see what people were looking for and where.

Analysts and programmers are currently working on how to expand on the simple examples above using search terms as wider indicators, a system called ‘sentiment analysis’ looks particularly promising.

This form of analysis looks at terms used during on line communication and categorizes them in terms of their sentiments. The logic is that in an area that is prospering terms will be generally positive, but in an area that is threatened by demise, such as the closure of industry or other societal problems, the terms will differ. This is not dissimilar to the conversation analysis sociologists use to obtain a person’s own sentiments about their position in life, with their true feelings reflected in the terms they use without thought. The hope is that an accurate analysis of this type might signal unfolding problems before they become a reality so that action can be taken in specific areas to avoid social breakdown.

I have addressed these issues in more depth on the Bassetti Foundation website, but want to conclude by saying the following; in my posts I have often raised the issue of data collection as a problem, and collection of personal data for advertising or any other purpose for that matter does raise serious ethical issues, but here Google et al could be sitting on a mine of extremely useful and possibly globally important data if the technology and political will is developed to use it correctly.

Are user generated reviews reliable?

The current March to April issue of National Geographic Traveler magazine has an article about user generated reviews that raises some interesting issues. I looked at the commercial invasion of the blogging world in one of my previous posts, but here author Christopher Elliott raises the issue of interested parties posting positive comments about their own businesses and negative ones about others, raising their star status and damaging others.

I was particularly drawn to this article because it addresses a problem faced by frequent travelers, and having spent a couple of years all in all on the road with all of my worldly belongings on my back I can relate to the problem.

Local offering help to lost travellers

Ask a local for advice and he looks it up on his iPhone

When you are out there alone, who better to ask for advice than fellow travelers? Many look to the internet for recommendation, and here lies the problem, lack of competition. There are really only 2 large websites of reference, Yelp and Travel Advisor, so it is relatively easy to either build up a profile or destroy somebody else’s using different user names and computers.

Within the industry an understanding of the problem is widespread, and both companies named above defend their positions stating that they have vetting procedures to catch out the bogus reviewers. It is very telling however that after a British Advertising Standards Agency investigation, Trip Advisor changed its slogan from “reviews you can trust” to “reviews from our community”, the implications are obvious.

The broader implications are vast too, many people read reviews before choosing a dentist or a school for their kids, and a bit of underhand behaviour could easily destroy somebody’s reputation.

My personal opinion is that these problems are representative of wider issues of internet governance. They are essentially born out of monopoly, the democratizing power of the internet and peer to peer communication usurped by business interests and competition. An infiltration of commercial interest into a non commercial ideology, that of offering advice to someone who finds themselves in a position that you were once in, becomes unreliable.

Scientists grow artificial meat

In my work at the Bassetti Foundation I have written extensively about food, its production and how technology has entered and continues to encroach on everyday provisioning.

I wrote an article about how milk from cloned cows entered the food chain last year, and then how some cloned cows went missing in Scotland, probably ending up at the butcher’s shop and on to somebody’s table, but this week I let you in to a story that goes much further.

A very fat cow

A fat cow

Scientists in Holland have managed to grow meat from stem cells and later this year aim to have enough to make a hamburger. I don’t know if they plan to eat it though, it will cost about $300 000 to produce. Although this sounds abhorrent as well as quite expensive, they have good reason to try it. Many experts believe that current food provisioning techniques are non sustainable. As the population grows more people require more food, but leave less land to use in its production, and this presents one problem. The second issue is that large scale meat production is one of the biggest carbon producing industries on the planet.

Meat production is also physically very damaging for the environment, there have been many articles written about beef production in Argentina and Brazil causing problems for the rainforests. Meat production is also inefficient. In order to produce meat (for example beef again) you need to feed up a cow, and cows eat a lot. The protein gained from the cow is equal to 15% of the protein it eats, so you have to feed a herd of cows 100 tons of protein for every 15 tons produced.

So could this new type of production help feed the planet? It will certainly have to overcome a few hurdles, production price will have to come down obviously and I imagine protests and general dislike of the idea, but OGM is everywhere nowadays (unmarked in the US I believe) and you can get used to anything if you try. The point is that it can be done, the technology exists and once done on mass it will be done cheaply.

If you are interested in learning more take a look at the article on the BBC website, my interview with Prof Andrew McMeekin of the University of Manchester Business School on the subject of food provisioning and my other meanderings on the Bassetti Foundation website.