Can researchers bring segments to life with AI chatbots?

Imagine you've conducted a meticulous segmentation study, carefully uncovering distinct customer groups with unique needs. You've delivered the insights and crafted a road map to success for your client, yet a lingering question remains: How can you ensure that these valuable insights permeate the entire organization and truly drive strategic action?

To achieve this, we often rely on workshops involving different business units and design materials such as posters, stickers or infographics. These tools help make the segments tangible and present the results in an easily understandable manner, thereby making them accessible to a broader audience within the company.

However, given the rapid development of generative AI and its proliferation across the market research industry, my team started to wonder how and if we can leverage this tool to help socialize segmentation insights throughout all levels of an organization. We were particularly inspired by the introduction of custom GPTs by OpenAI as part of the ChatGPT 4.0 update. Instead of relying on traditional methods, imagine simply asking an AI chatbot – trained on proprietary data and insights – for the answers to all your research questions.

The goal with this experiment was to train such a bot using data from a market research study so it could actively engage users and provide insightful answers about the study's findings. We envisioned three distinct chatbot prototypes, each aiming to bridge the knowledge gap for different user needs within a company.

‍

‍

1. Persona bot: Fostering empathy through interaction.

Imagine stepping into the shoes of a customer segment! The "persona bot" facilitates this experience by allowing users to directly interact with the defined segments using qualitative data from the study. By simulating real conversations based on the personas' perspectives, the goal of this bot is to foster a deeper understanding of each segment's motivations and behavior.

2. Knowledge bot: A central hub for uncomplicated access.

Think of the "knowledge bot" as a one-stop shop for all study data. This bot serves as a central repository, equipped to answer any questions related to the segments and the overall study with informed, data-driven responses. It leverages both quantitative and qualitative data, aiming to provide direct, user-friendly access to information, eliminating the need to navigate bulky reports or presentations.

3. Strategy bot: From insights to action planning.

The "strategy bot" goes beyond data presentation. It acts as a brainstorming partner, equipped to answer questions across various business areas such as marketing and product development. By leveraging its understanding of the segments, this bot can provide strategic support for planning and decision-making, ensuring initiatives are directly aligned with corporate strategy.

Testing AI bots with fictional segmentation study  

To test these concepts, we conducted a simulated segmentation study for a fictional client, "VacaVista," an online travel platform. We generated extensive data and documents mimicking a real study, including a detailed questionnaire, Excel data sets with survey results and qualitative persona descripts to train the bots, and segmenting VacaVista's customer base into five distinct groups based on factors like travel planning intensity and budget. One of the segments, the "Enthusiastic Deal Hunter", for example, enjoys planning trips meticulously and is often in search of the best deals. Through experimenting with different combinations of data and refining bot instructions ("prompts"), we gained valuable insights into the potential of the three bot concepts.

‍

‍

The persona bot offered a glimpse into the segment, but not the whole picture

In the early development phase, the persona bot provided initially authentic-seeming responses. We meticulously trained it using qualitative data (persona descriptions) and precise instructions to mirror the personas' characteristics. Responding in the first person initially enhanced the impression of interacting with a "real" representative of the segment.

However, as conversations progressed, the bot's responses often appeared one-dimensional. For example, the bot representing the "Deluxe Time-Saver" segment tended to emphasize luxury aspects in nearly every response. While this is a significant characteristic of the segment, the responses seemed stereotypical and less human; it increasingly felt like the bot generated clichéd answers. The insights we typically gain from conversations with humans sometimes lead to counterintuitive discoveries and help us form new hypotheses – a phenomenon we couldn't replicate in conversations with the bot. The bot failed to convey a balanced and nuanced picture of the segments, leading us to decide to shelve this idea for now.

‍

‍

The knowledge bot delivered facts at our fingertips, but accuracy concerns remain

This led us to consider whether it might be more promising to task the bot with presenting numbers and facts. After all, these can be objectively verified, providing a reliable basis to assess the bot's performance.

In developing this bot, we focused primarily on quantitative data from the segmentation study, supplying the bot with Excel tables. After experimenting with different formats that the bot could best understand, we succeeded in enabling it to provide numbers and tables upon user request. For example, it could answer data-driven questions about the segments' travel and booking behaviors, incorporating numbers to support its statements. It could even reproduce entire tables and summarize key points in its own words, which could be particularly useful for those less comfortable with numerical data.

However, testing revealed recurring issues: The bot occasionally incorporated incorrect numbers into its responses, seemingly without a discernible pattern, even when explicitly instructed not to fabricate data. Additionally, there were frequent system errors and long wait times due to the bot's struggles in interpreting large data sets – a common challenge with quantitative segmentations.

Considering the rapid evolution of AI technologies, we hope such issues will soon be resolvable. Nevertheless, the presentation of incorrect data was a significant factor that deterred us from further developing the knowledge bot as a customer tool at this stage, as we cannot consistently guarantee the quality and reliability of its responses.

‍

‍

The strategy bot unveiled new insights through multi-layered responses

Our most promising finding came from the strategy bot. This bot, equipped with comprehensive knowledge of the segments, aimed to enrich brainstorming and the development of new strategic initiatives. It demonstrated its ability to answer strategic questions across various business areas, such as marketing and product development, considering segment-specific nuances. For instance, it provided compelling recommendations on segment prioritization that closely aligned with our own professional assessments. Even in creative contexts, the strategy bot offered plausible responses and served as a valuable source of inspiration. When asked, for example, to create concise advertising messages that resonate best with the segments, it generated creative and fitting ideas tailored to segment preferences.

‍

‍

Are AI chatbots the future of market research?

Our experiment with segmentations suggests AI chatbots worth the chat. While some early ideas need a bit more work, the strategy bot showed promise as a brainstorming buddy for strategic planning. Overall, AI chatbots could become an effective means to promote the widespread dissemination and acceptance of market research data, especially segmentation data. However, as we all know, their responses still need to be critically evaluated. I advise caution when using them to generate empathy for segments or as reliable sources of study data … they are just not quite there yet.

‍

Originally published on Quirks.com

‍