TV Viewing in India: Comparing Human Users and Generative User Profiles (GUPs) on UX4

Introduction:

The UX research landscape is undergoing a seismic shift. Conventional methods, constrained by time and costs, struggle to keep up with the pace of development seen in today’s organizations. Why should agile product teams, building iteratively at speed, wait 3 months for a research project before getting actionable insights?

Organizations are addressing this gap by adopting rapid AI-enabled approaches to conducting user research including the use of synthetic data produced by AI-generated users. This comes with the promise of avoiding the quagmires of recruitment and scheduling, saving several weeks and thousands of dollars on UX research. While this is a promising approach, one question remains: Can synthetic data truly match the depth and nuance of human insight?

To demonstrate the accuracy of GUPs compared to human participants, we conducted an open-ended, qualitative study on the dynamic Indian TV viewing space.

The Media Battleground in India

India’s television and video consumption market is undergoing significant transformation with the battle between Direct-to-Home (DTH) satellite services and Over-The-Top (OTT) streaming platforms. This convergence of 2 culture-defining technologies creates a compelling research topic.

The goal of the study was to uncover the evolving behaviors, motivations, and contextual factors influencing TV and video consumption habits among Indian households and individuals. Working with researchers at Human Factors International, we designed the interview protocol for the study, covering the social, contextual and relational aspects of this complex topic.

Factors affecting TV consumption are not just geographical (urban vs. rural) but highly generational, making it a thorough test of GUPs’ fidelity against human behavior. Our study was structured around four distinct demographics:

Partition Generation (Age 62–81): This cohort lived through the aftermath of India’s independence and the Partition of 1947. These events shaped their values with a focus on stability and tradition.

Transition Generation (Age 42–61): Grew up during increasing economic opportunities and the lead-up to India's liberalization in the early 1990s.

No-Strings Generation (Age 29–41): Came of age during a period of globalization, liberalization, and technological innovation.

Gen Z (Age 13–28): Raised in a digitally connected, rapidly evolving India. Influenced by global trends and access to information.

Both of the studies were conducting using UX4’s AI-enabled interview moderator. The human study was conducted using an asynchronous WhatsApp-based protocol, based on our co-founder Apala’s PhD dissertation demonstrating the reach and efficacy of such approach, as well as other papers demonstrating similar results. The resulting data was analyzed using a combination of LLM labeling, and hierarchical clustering, similar to the approach used in this paper.

Configuring a Study with Human Participants

UX4’s AI interviewer engaged with 24 human participants across 7 Indian cities via Whatsapp. It took approximately a week to setup the protocol and recruit participants. It took 2 days to complete all the interviews, and another 2 says to process and synthesize the findings. Overall, the study took about two weeks to complete.

Configuring a Study Using GUPs

For the other half of the comparison, we created 24 GUPs which aligned with the high-level demographics of our human participants, and were based on prior research conducted by HFI with similar users. These GUPs were interviewed using by the AI moderating, using the same protocol. Preparing the GUPs took approximately a day, and then the actual simulations were run in a matter of minutes.

Findings:

Key Insights from the TV Viewing Study

Here are 3 key insights which emerged from the human participant interviews:

Cost, Convenience, and Bundling Shape Platform Loyalty:
Affordable, flexible bundles drive adoption and loyalty across generations. Users across generations equate more channels with better value, although a minority prefers control and flexibility (only pay for what they consume). Rather than managing multiple connections and confusing channel packs, participants prefer a single, bundled solution that serves the entire household — fewer bills and less management equal better value. 
Gen Z participant: For DTH we have to take a DTH connection for every room which becomes expensive… OTT you can pay once and all five members are using it on the phones in their personal room TVs, hall TV, laptop - anywhere.No-strings Generation participant: Because I have installed Airtel Xstream Fiber - they give a combined offer with DTH and it is very cost-saving.Partition Generation participant: I wouldn’t have to pay for it separately - it would be convenient to pay for it at once.

Generational Shifts Highlight Hybrid Viewing Patterns:
TV anchors communal viewing, while younger users favor OTT/mobile. DTH remains a trusted and stable content delivery method for most users, particularly among older generations. However, functional frustrations and lack of smart integration are pushing users - especially younger ones - toward OTT. The future of DTH may depend on smarter and flexible ecosystems that bridge traditional TV with modern digital expectations.
Partition Generation participant: We have taken a combo plan which offers internet and TV channels as well as few OTT channels, so it's more convenient to watch through Jio’s service.Transition Generation participant: If I miss an episode on TV, I try to find it on Hotstar or YouTube.Gen Z participant: Yes, I would prefer to watch on OTT because if someday I couldn’t watch on DishTV at that specific time, I can watch on OTT at any time.

GUPs Achieve 91% Thematic Similarity

After looking at interview data from the 24 GUP interviews, we found that they consistently emulate the core conversations, motivations and preferences of human participants. They discuss the same topics and exhibit similar patterns of likes and dislikes.

We applied a two-step approach (manual and algorithm-based) to validate this similarity. UX4’s AI analysis of the human interviews established 12 themes (using the algorithm described earlier) served as our benchmark for assessing fidelity.

Approach 1: Manual Theme-matching

We first tested how many themes from the AI analysis of GUP interviews directly aligned with the themes elicited from the human benchmark. Themes emerging from the GUP interviews were "highly similar to" and "hold the same meaning" as 8 of the 12 benchmark themes. These direct matches account for a thematic similarity of 67% . For context, here are 2 examples of direct matches:

To understand the remaining themes that emerged from the human study with no direct matches in the GUPs analysis, we performed a manual data deep dive into the GUPs transcripts. We discovered that GUPs did talk about 3 of the 4 remaining themes. For example, "Cost and Bundling Considerations" was a key theme amongst human participants. Our review confirmed that GUPs talked about this too:

By combining the 8 direct thematic matches with the 3 themes found via manual inspection of the GUP transcripts, we establish an overlap of 11 out of 12 themes. This approach results in a thematic coverage of approximately 91%.

Approach 2: Similarity based on Embedding Distance

To replicate this result, we also validated using a more deterministic statistical approach. Word embeddings were calculated for each of the themes from both the human and GUPs studies. Cosine distance was used to estimate the similarity of the themes under each of the conditions. Using this approach we again found that 11 of the 12 (91%) of the themes covered in the human study were also surfaced in the GUPs analysis.

What else can we learn from GUPs?

While achieving 91% similarity proves GUPs are a reliable source of user knowledge, we were also interested in finding out what we could learn from GUPs that may be more difficult to obtain from human users. We learned that this divergence holds significant strategic value by providing high-potential innovation opportunities.

GUPs are Expressive

Compared to most humans, GUPs are consistently better at articulating why they like, dislike, try or recommend something, providing product teams with clearer guidance.

GUPs Uncover Opportunities for Innovation

GUPs are also better at expressing emerging needs and desires, i.e; what excites them, what they wish existed. This led to several unexpected, creative ideas coming out of the GUPs. However, these ideas must still be validated through discussion with real human participants.

GUPs Should Be Deployed Responsibly

While GUPs offer exciting potential exceptional ROI, we acknowledge their current boundaries.

GUPs are trained on internet data that can be 6-8 months out of date, meaning they may miss the very latest user trends.

Fidelity may vary for highly niche or less digitally active populations due to limited public data.

Human oversight is critical in the responsible deployment of GUPs for UX research (provided by the many expert researchers at HFI).

Research done with GUPs must be validated through interactions with real human participants. The goal is to make real participant research more targeted and efficient.

Research with AI simulations must consider the ethical implications. In this we are guided by our three principles for conducting responsible UX research with AI.

Conclusion and CTA:

91% Accurate, Exponentially Faster and Economical. GUPs make Business Sense

Both manual and statistical methods confirmed that GUPs achieve a high degree of thematic similarity to data elicited from human participants. Generating such quality of insight at a fraction of the time and cost spent on conventional research makes GUPs a great business investment.

A traditional UX research study of this scale can take 2 months or more to complete. We completed the human study in 10 days by leveraging UX4’s AI Moderator and WhatsApp based interface. This comparison showed that you can also use GUPs to derive similar results within minutes. Both approaches are supported natively by UX4, from protocol design and interviewing live participants to GUP generation and analysis, demonstrating its capability of handling end-to-end UX research leveraging a variety of modalities and knowledge sources.

Beyond the increase in speed, the real value lies in shifting from a reactive research model to a proactive one. This study demonstrates a clear path to move beyond project-based research that lags behind development. Instead, insight becomes an "always-on" resource, allowing your teams to de-risk decisions and validate concepts within a single sprint. This makes user-centricity a daily practice, not a once in a while milestone.