In today’s corporation, the role and value of Market Research is often questioned when data is collected and analyzed without solid business context. Herein is a key difference between traditional Market Research and Market Insights; Market Insights takes into account the entire body of available information while Market Research often ignores any information beyond the purview of a particular unified methodology.

Recent years have seen the rise in titles such as “insights and analytics” attached to the Market Research function in order to enhance and refocus the role of these groups. In other organizations, traditional Market Research departments are being completely consumed into other business units because the value of a stand-alone Market Research function is not apparent to upper management. In order avoid such a fate, the insights team must start by understanding the basic anatomy of what makes an insights department, where they are in terms of a baseline, and how to most effectively achieve the status of a trusted advisory team.

Understanding the Anatomy of a Successful Insights Function

Figure 1: The Three Rings of a Successful Insights Function

The solid inner ring of Figure 1, entitled Information, encompasses all available information within the organization. The inside dotted line represents the more traditional Market Research function whose charter is to conduct primary research and acquire secondary Market Research data. The third major source of information, represented by the upper left hand circle, revolves around the internal enterprise systems that track things like finance and billing, customer marketing databases, and operational systems (i.e. the company’s traditional ERP and CRM systems). The lower left hand circle represents external data provided by organizations such as Google, the US Government; or the massive amounts of data provided by web scraping. The lower right circle represents competitive intelligence which gets into much of the companies benchmarking protocols. Finally, and perhaps most importantly, there’s the institutional knowledge that may not be captured in the other knowledge areas. This institutional knowledge will often be the most important since it can speak to what’s been tried in the past and the organizational politics surrounding particular decisions. This institutional knowledge can be anything from an executive’s team vision, to competitive insights dormant in the sales force.

The second solid ring represents the extraction of true insights, or wisdom, from the inner ring of information sources. The provision of insights is really the domain of the insights analyst. Since there are myriad information sources, someone who is skilled at looking at multiple sources of data and putting it into a coherent story is critical to the organization.

Ultimately, the best insights in the world are worthless if nobody is listening. The outer ring represents a continuous advancement of the company’s willingness to listen to the Market Insights. This is something that starts at the top of the organization and must be culturally engrained across all departments. At the same time, it is the responsibility of the successful insights department to promote such a culture by providing market advantage through their insights, and making sure those at the top of the organization have bought into the insights culture.

Understanding the Levels of Departmental Maturity

Depending on the organization, the insights department will generally be advancing through, or moving back through, one of the following three stages. These stages, in increasing order of maturity, are a commodity supplier to the marketing or purchasing department, a respected decision support team, or at its highest level, a trusted advisory team. Given that the three stages represent a continuum, many departments may in fact be present in two stages simultaneously.

Stage One- Commodity Supplier

Though the insights function is an internal department in this first scenario, the Commodity Supplier is more of an order-taker than a true part of the company’s fabric. Often reporting to the marketing department, the function is led by a mid-level manager (but in some cases as high as the Vice-President level or the Senior-Vice-President level). Some organizations, unfortunately, are receding into this stage as cost pressures or perception of low-value Market Research has engulfed their thinking.

Evidence that a department is in Stage One is a subordinate relationship with marketing and likely a subordinate relationship with the purchasing department as well. There is no true executive sponsorship and for this reason decisions are often governed by cost control measures. The research staff is usually less experienced in research methodology and lower paid relative to industry average. The staff is usually compensated on a fixed salary basis with a very low portion of the compensation, if any, based on performance. The group as a whole has a lower headcount and a lower budget than its industry peers in Stage Two or Stage Three. This creates an environment that lacks support and managers are usually stretched between many projects. There is really no budget authority emanating from this group given that they have no real management support. In fact, they are generally the last to know about new initiatives or departmental cuts since they do not take part in regular stakeholder meetings. If they are invited to important meetings, it is usually on the tail end of the initiative process.

The result is that research is conducted by the numbers and specifications are written with little or no input from the actual business decision makers. The research is usually tactically focused and low-value questions are asked of the market; whereas high-value questions leading to game changing insights are only peripherally addressed or left out all-together. The findings are written simply as a recasting of what is found in data tabulation tables without any synthesis of information or value-added industry insight. In the silo of the Stage One department, true findings are rare, and truly useful recommendations are practically non-existent.

Stage Two- Decision Support Team

In Stage Two of departmental maturity, the insights team turns into a Decision Support Team and begins to garner more confidence and respect. At this stage, key executive sponsors for the insights function have arrived on the scene, and suddenly people are much more interested in information-based decision making. In this environment, the insights department becomes needed, though not always wanted.

 Evidence that suggests the insights department is in Stage Two is that the group is seen more as a peer to other functional groups such as marketing, purchasing, IT or operations. At this level, the insights managers are more credible and a bit more seasoned. They may have line management experience and they are likely well versed in either information analysis or in the industry in which the company resides. Sometimes they are versed in both, which makes them more valuable. Importantly, the insights staff is better compensated and often has a portion of compensation tied to performance (as evaluated by internal stakeholders and management). While headcount may or may not be expanded from stage one, it is generally adequate. Unlike a Stage One organization, the group has input into budgets and supplier choice is not driven by ‘cost only’ considerations.

From an output perspective, the portfolio of studies in the Stage Two department tends to be less tactically focused and more strategically focused. By virtue of being included earlier in the process and having direct access to the business team decisions, the insights team produces demonstrably good reports where most, if not all, of the important business questions are answered. Information is synthesized with previous research and internal sources, and the insights are put into an industry perspective. Recommendations are still primarily the domain of the business manager, but the insights manager provides enlightened perspective for that decision.

Stage Three- Trusted Advisory Team

The third stage of maturity, the Trusted Advisory Team, provides foundational guidance for the organization. At this stage, the CEO and the executive staff have completely bought into the value of the insights function and information-driven decision making. This vision, in turn, permeates the organization. The Market Insights function has become a driving force in the company culture and a source of competitive advantage in the marketplace.

Evidence that suggests the company has achieved Stage Three status is, for example, the value of the function being mentioned in the annual report and shareholder meetings. The CEO and other executives begin to set corporate mission, objectives, and policy based on Market Insights. In Stage Three, Business Intelligence and Market Insights become somewhat interchangeable with a subtle distinction. The insights department passes things through purchasing as a mere formality and sits in stature with teams such as finance or marketing. The powerful C-level managers now see the leaders of the insights function as necessary allies within their ranks.

In Stage Three, the insights managers are well-compensated, performance-driven, and top-tier analysts. These individuals are trained in the science of data analysis and the art of storytelling; they know the organization and the players, and they likely have a certain gravitas about them. Indeed, these insights managers are the uberanalysts to whom we refer throughout the book. Formal processes are in place throughout the organization so that the insights managers are a part of each important meeting during the initiative process, and they are seen as a trusted advisor for each level of the organization. The group as a whole stems from the profile of these individuals, and the function is seen more as an investment center rather than as a cost center.  Resources are easier to come by and budget requirements are virtually always met.

The resulting research portfolio is heavily weighted to strategic issues and considerations. While there is tactical research being conducted, it is always as outflow from strategic initiatives. All the right questions are asked of the internal databases and of the customers directly. In fact, the questions themselves are at the very heart of the corporate strategy. Information and insight is grounded in multiple organizational sources and reports are never presented as reams of data with self-evident captions. Rather strategic guidance is rendered on things such as opening new markets, increasing current share of customer wallet, or increasing margin through differentiation. For the Stage Three insights function, the voice of the market acts as a lighthouse for organizational strategy.