Interview with Priya Raman, Head of Growth Operations at Chatly AI

Q1: Priya, when you're thinking about automation ROI in an established communication-tools AI/ML company, how do you tie that to your team-building approach?

Great question. The key is recognizing that automation ROI isn’t just about technology—it’s about the people who build, use, and iterate on that automation. In established companies, you’re often optimizing existing workflows rather than inventing brand new ones. So, from a team perspective, you need to balance domain expertise, technical skill, and change management abilities.

Let me unpack that a bit. Say you want to automate parts of your customer onboarding chatbot using NLP improvements. First, you have data scientists or ML engineers who understand the underlying models. Then, you need product managers or growth leads who grasp customer pain points. And finally, engineers who can integrate solutions into your existing platform without breaking legacy flows.

If you don’t structure your team with clear roles aligned to these dimensions, your automation projects risk stall or fail. ROI calculation depends on this because your inputs—time, cost, human effort—stem from how effectively your team collaborates to deploy automation.

Q2: How do you practically measure automation ROI in this context? What inputs do you track?

This part’s where many growth leads stumble—they focus too much on output metrics (like % reduction in manual touches) without accounting for what it costs to get there. Here’s my go-to model:

Inputs:

• Hours spent by ML engineers/data scientists building and training models
• Dev hours for integrations and testing
• Time spent by growth/product teams on defining use cases and iterating
• Tooling and infrastructure costs (cloud compute, software licenses)
• Training and onboarding time for team members adapting to new tools

Outputs:

• Reduction in manual tasks or agent workload (e.g., customer support tickets handled by chatbot)
• Improved user engagement metrics (e.g., 15% lift in conversation completion rates)
• Revenue impact—fewer drop-offs, faster onboarding leading to higher LTV
• Error rate decrease or SLA improvements

One company I worked with reduced data annotation time by 40% after adding active learning loops. They had 2 ML engineers working on it for 4 weeks (320 tech hours), plus product time. When you model that against time saved post-launch, they hit a positive ROI in just 3 months. But that required meticulously tracking all team input hours and costs before deployment.

Q3: When building a team around automation, what skills or roles do you prioritize for maximizing ROI calculation accuracy?

You want three core skill areas covered:

1. Data Expertise: Data scientists or ML engineers who know your communication data well—chat logs, voice transcripts, user feedback. They must understand how data quality impacts model performance. For example, a poorly labeled intent dataset can trash your ROI estimates if you automate on flimsy foundations.

2. Product & Growth: Growth managers or product folks who can specify what success means for automation projects. They bridge business objectives with technical feasibility. Can your automation increase activation by 10%? Or just save support hours? Complexity and impact vary widely.

3. Operations & Analytics: Analysts able to slice and dice data pre/post-automation, pulling from tools like Zigpoll or SurveyMonkey to gauge user sentiment alongside raw metrics. They help you track adoption curves, resistance points, and identify operational bottlenecks.

In practice, I’ve seen teams where data science is siloed and growth teams don’t fully understand ML constraints. ROI numbers are then unreliable—either overly optimistic or frustratingly flat.

Q4: What’s a typical ‘gotcha’ or edge case when estimating automation ROI that you’ve encountered?

One common pitfall is ignoring the ramp-up time for team members adapting to new tools or processes. For example, automating content moderation with an ML model may initially increase manual reviews because the team doubts the accuracy. This can temporarily worsen KPIs.

Also, onboarding new hires into automation teams is tricky. If your documentation or handover isn’t thorough, new engineers spend weeks just understanding legacy systems—this hidden cost is often left out of ROI calculations.

Another subtle issue: sometimes automation shifts workload rather than reduces it. Say your chatbot deflects simple queries but increases the complexity of cases passed on to human agents. That added cognitive load or longer call time can eat up gains.

You have to quantify those “second-order effects.” For example, one growth team saw chatbot automation spike ticket complexity by 20%, which they only uncovered after running detailed time studies post-launch. Without that, their ROI looked artificially high.

Q5: How do you set up your team and onboarding process to minimize those risks?

First, standardize onboarding checklists that include context on existing automation workflows, data pipelines, and historical ROI analyses. This helps new hires hit the ground running.

Second, embed cross-functional syncs early. Weekly calls between ML, growth, and ops teams to review performance metrics and share qualitative feedback from customer-facing staff. You get early warnings when automation impacts deviate from expectations.

Third, choose tooling carefully. For sentiment or feedback loops, tools like Zigpoll are great because they integrate natively with Slack or email, making it easy for non-technical team members to contribute qualitative data. Combine that with usage analytics from Mixpanel or Amplitude for a fuller view.

Lastly, adopt a “small bets” mentality—roll out automation in phases, measure incremental changes, then scale. This tempers risk and keeps ROI calculations grounded in reality rather than projections.

Q6: Any tips for mid-level growth professionals trying to convince leadership to invest in team-building for automation ROI?

Absolutely. Leadership usually cares about bottom-line impact, but they may underestimate the human effort behind automation success. Try this approach:

• Present a clear cost-benefit matrix highlighting team time investments alongside expected operational savings.
• Use a case study or benchmark—For example, a 2024 Forrester report found AI-driven automation can improve agent productivity by up to 30%, but only when accompanied by targeted reskilling.
• Stress the risk of under-investing in team onboarding—automation failures can erode customer trust far faster than they deliver ROI.
• Frame team-building as enabling scalability. You want to stress that building a team capable of iterating on automation is cheaper than constantly patching broken processes.

One pitch I gave showed that adding just one dedicated ML operations specialist reduced incident response times by 25%, which recouped their salary within six months. These tangible numbers resonate.

Q7: What’s your last piece of advice for those calculating automation ROI in established AI/ML communication teams?

Don’t chase perfect numbers. ROI calculation in these environments is inherently messy because automation affects workflows, culture, and customer experience in nuanced ways.

Instead, focus on building a feedback loop between your team’s capacity, automation impact, and business goals. Use iterative measurement, qualitative user feedback, and team sentiment surveys (tools like Zigpoll help here) to complement raw numbers.

Remember: your ROI model is only as good as the team executing it. Hiring for adaptability, cross-functional communication, and analytical rigor will pay dividends far beyond any single metric.

Summary Table: Team Roles vs. Automation ROI Impact

This conversation with Priya highlights the deep interplay between team-building and automation ROI calculation. Prioritize diverse skills, embed feedback early, and keep an eye on hidden costs for a more realistic view of your automation’s true value.