Customer Login  |  

From Category Archives:

Data Harvesting

by Shyamali Ghosh

Frei 4

A glimpse of old-school data collection and dissemination as seen on a tour through the Sprudelhof, Bad Nauheim's turn of the century bathing complex, one of the International Competitive Intelligence Conference 2012's social events.

Competitive intelligence (CI) is a craft used primarily by large, multinational corporations—think Nestle, IBM, and Novartis—to steer strategy. Keeping an eye on technology developments, competitor sales data, or the movements of key industry players are all common CI practices and, while many may confuse CI with corporate espionage, most of the information analyzed is freely available via open sources.

Last week dozens of these in-house competitive intelligence practitioners gathered in Bad Nauheim, Germany to attend a conference held by the Institute for Competitive Intelligence. I was there to understand how to improve IEI’s CI research practices, and here is my brief assessment of the current state of this important market for information services.

The amount of data gathered and analyzed by companies depends on their size and industry, but multinational consumer goods companies are among the largest consumers of CI services. These firms constantly monitor and analyze multiple streams of real-time data including social media conversations, ecommerce data, and competitor web site activity by using high-end data extraction tools and crowdsourcing (two of IEI’s practice areas). In other cases, for example to gain detailed information on emerging new technologies, only in-depth primary research with industry experts and think-tanks will do. In either case, the design of the data-gathering process remains the key to the accuracy and timeliness of the final analysis. Key questions include: Where is key information available? How can we get it quickly and cost-effectively? How do we feed it into our analysis routines?

Elvis Presley Platz

Elvis Presley Platz, Bad Nauheim, Germany, just across the street from the venue for the International Competitive Intelligence Conference 2012.

The data is out there, and there’s more of it every day, so the CI practitioner’s challenge has become how to gather, standardize, sift, and analyze all of it fast enough to give their employers a competitive advantage. In many ways the practice of CI is similar to the editorial side of the publishing industry, which exists to keep their customers on top of their industry or profession’s latest trends. In fact, there are several large erstwhile publishing firms like IHS that have very successful high-end industry intelligence services and offer custom consulting services, too. When the project is just too sensitive to trust with outside vendors, it can only be handled in-house. That doesn’t mean, however, that all the raw data gathering and structuring can’t be outsourced or crowdsourced, and we expect to see a lot more of this in the near future.

{ 2 comments }

posted by Shyamali Ghosh on April 2, 2012

As the “big data” juggernaut* continues to build momentum, it is about time to pause and reflect on those things that are holding us back from realizing the true potential of data integration, overlays, and analysis. Mark Miller posted a great piece on MediaPost (“Three Steps To Dealing With Data Paralysis,” Jan 31, 2012) that points out a few of those issues and how to circumvent them.

  • Favor smart data over big data: “Data issues like missing values, missing linkages and data anomalies can impede your ability to harvest data and move the business forward…. have a plan to deal with quality issues that will otherwise make your data dumb.” Hallelujah.
  • Use analytics to mine your smart data: “…identify the 10-15 most important things you need to know… Focus on the information that helps you determine…how you can create value.” Setting the right data priorities is, of course, key.
  • Create a roadmap using data and analytics: “…conduct an audit of the processes, systems, tools, and talent within your organization. Identify the gaps. [This audit] should be connected to harder metrics like sales, revenue and profit.” There a couple of good points here: 1) If you don’t have the talent or bandwidth in-house for the data work then go get it on the open market; and, 2) Always focus on “real” numbers — confusing “reporting” with “analysis” is a fatal mistake and focusing on hard metrics (not forgetting that cost reduction is among those) is essential.

Miller comes from the CRM world, but what he is saying is relevant to many others in the world of creating and managing data. We may all be awash in data and have decades of experience under our belt managing it, but it’s never to late to go back to basics and make sure that the foundation for our databases is well laid, our cost of ownership is low, and we are getting every dime of value out of what we have built.

*Derived from various Indian words for a very large, portable, wooden temple.

{ 0 comments }

posted by Shyamali Ghosh on February 6, 2012