Improving productivity and Transparency of a Service Business with AI

Is Remote Work Sabotaging Your Team’s Productivity? Here’s How We Solved It With AI

In the wake of a global shift towards remote work, companies worldwide are grappling with maintaining productivity and ensuring effective collaboration among distributed teams. How do you sustain visibility into your team’s work without resorting to intrusive monitoring? With attendance declining and a glaring lack of traceability in a remote environment, we knew we had to innovate. Here’s an in-depth look at how we created a comprehensive data gathering and Large Language Model (LLM) platform to track and enhance employee productivity.

Level Required: Programming Skills Needed

This solution requires moderate to advanced programming knowledge and cannot be accomplished with no-code tools. You’ll need experience with APIs, database management, AI model integration, and Slackbot development to replicate this setup.

Navigating Remote Work and Lack of Traceability

Transitioning to a fully remote setup brought numerous benefits, including flexibility and access to a broader talent pool. However, it also introduced complexities in monitoring employee engagement and productivity. Traditional in-office monitoring methods were ineffective, leading to a decline in attendance and a lack of clarity on project progress.

Key Issues We Faced:

1. Decreased Attendance: Without physical presence, tracking attendance became challenging.
2. Lack of Visibility: It was difficult to ascertain who was working on what, leading to potential overlaps and inefficiencies.
3. Data Silos: Information was scattered across various platforms, making it hard to get a unified view of team activities.

Realizing the need for a robust solution, we embarked on developing a system that could seamlessly integrate data from multiple sources and provide actionable insights.

Our Solution: Building a Comprehensive Data Gathering and LLM Platform

To address these challenges, we designed and implemented a data gathering and LLM platform tailored to track employee productivity effectively. This solution involved aggregating data from various communication and project management tools and leveraging AI to interpret and present this data in meaningful ways.

Identifying the Data Sources

We began by pinpointing where valuable information about our team was generated and stored. By focusing on key platforms, we ensured comprehensive data coverage.

Platforms We Focused On:

• Slack
  • We primarily capture presence data (i.e., who is active and when) to understand team availability.
  • Slack API Docs: https://api.slack.com/
• Google Workspace (Gmail, Calendar, Drive)
  • Essential for emails, scheduled meetings, and shared documents.
  • Google API Docs:
    • Gmail API: https://developers.google.com/gmail
    • Calendar API: https://developers.google.com/calendar
    • Drive API: https://developers.google.com/drive
• Project Management Tool (Jira)
  • Tracking of ongoing tasks and project progress, offering insights into individual contributions.

Each platform contained unique data points that, when combined, offered a holistic view of employee activities and project statuses. Consolidating these data sources helped us eliminate silos and create a unified repository for analysis.

Aggregating Data Using APIs

After identifying our data sources, we needed to centralize this information. This was achieved through APIs provided by Slack and Google, which enabled us to extract, organize, and store data automatically.

Leveraging APIs for Data Extraction:

• Slack API
  • We only retrieve presence data—i.e., when team members are online and available—instead of full message histories or file shares. This approach respects privacy while still providing essential visibility into team availability.
• Google API
  • We use Gmail threads, calendar events, and documents in Google Drive to track formal communications and scheduled activities.
• Jira API
  We pull the activity feed from Jira, capturing task updates and progress logs to gauge overall productivity and project health.

Building the Centralized Database:

• Data Organization: A schema designed to efficiently capture and query aggregated data.
• Security: We employ granular permission controls to ensure that only authorized users can access sensitive data as needed.

Designing the AI Agents (2)

A key element of our platform was the development of two specialized AI agents: the Query Agent and the Data Interpretation Agent. The decision to use two agents was driven by our need to divide responsibilities and ensure optimal performance in each specific task.

Why Two Agents? The Rationale Behind Our Approach

1. Separation of Concerns
   Each agent specializes in its unique function—query translation vs. data interpretation—simplifying development, maintenance, and scaling.
2. Enhanced Accuracy and Efficiency
   The Query Agent is optimized for natural language understanding (minimizing misinterpretation), while the Data Interpretation Agent focuses on data analysis and visualization.
3. Scalability and Flexibility
   Each agent can be independently updated and scaled to address evolving data sources or user needs.
4. Improved User Experience
   Users receive precise and prompt answers, thanks to a streamlined query-response cycle.

Example Scenario

When a user asks, “Show me the progress of Project Alpha over the last month,” the Query Agent parses the question, retrieves the relevant data from the database, and hands it off to the Data Interpretation Agent, which generates an insightful report highlighting milestones and potential delays.

Query Agent:

• NLP: Understands user queries in natural language.
• Query Translation: Converts user questions (e.g., “Who is working on Project X today?”) into precise database queries.
• Contextual Understanding: Maintains context, enabling deeper and more relevant responses.

Data Interpretation Agent:

• Data Analysis: Analyzes retrieved data to extract meaningful insights.
• Report Generation: Highlights key metrics like individual productivity, project progress, and team collaboration trends.
• Visualization: Generates charts and graphs for quick comprehension.

Building the Database Schema

We opted for a straightforward approach by storing all data in separate tables—one for users, another for projects, and so on—to keep the schema simple and easy to maintain. This modular structure allowed us to map Slack presence data, Google Workspace events, and Jira tasks into distinct, clearly defined tables. Although more complex relationships could be established later, starting with separate tables minimized design overhead and made it easier to evolve the schema as business needs changed.

Integrating the AI with Slack Commands

We integrated our solution directly into Slack—our team’s central communication channel.

Creating a Slack Bot:

• User-Friendly Interface: Enables queries and responses without leaving Slack.
• Natural Language Interaction: Users simply ask questions in plain English.
• Bot Functionalities: Instant queries, automated updates, custom commands for quick interactions.

Enhancing Collaboration:

• Transparency: Basic availability data encourages accountability without extensive data collection.
• Ease of Access: Users can quickly retrieve project or attendance info without toggling between tools.

Handling Slack’s 3-Second Response Limitation

Slack enforces a minimum response time of 3 seconds for command-based interactions to maintain system stability.

1. Why the Delay?
   Complex queries and in-depth reporting may exceed Slack’s time threshold.
2. How We Managed It:
   • We send an immediate acknowledgment (“Processing your request, please wait…”) to satisfy Slack’s response requirement.
   • Once AI processing is complete, we deliver a second, detailed response with comprehensive data.

This dual-message approach ensures Slack’s policy compliance without sacrificing the depth and accuracy of our AI-driven insights.

Developing AI Agents with OpenAI

Why We Chose o1-mini

We initially experimented with several larger language models, such as GPT-3 and GPT-4, but found that their higher resource requirements and associated costs posed significant challenges for real-time Slack integration. After testing various open-source alternatives as well, we ultimately chose o1-mini because it strikes a balance between performance and efficiency. Not only did it deliver sufficiently accurate natural language understanding and data interpretation for our use cases, but it also ran quickly enough to support on-demand queries without straining system resources or budgets. This lightweight model gave us the scalability and responsiveness we needed, making it the ideal choice for our productivity platform.

How Data Moves Between the Two Agents

Once the Query Agent completes its work of converting a natural language request into a precise SQL query, it executes that query against our database and retrieves the relevant records (e.g., rows of user activities, project updates, or Slack presence logs). The retrieved data is then serialized—usually in a lightweight format like JSON—to ensure a consistent structure for further processing. This JSON payload is subsequently passed to the Data Interpretation Agent, which parses the data, analyzes key metrics (e.g., productivity scores, collaboration levels), and generates a user-friendly summary or visualization. By using a structured handoff in JSON, we maintain a clean separation of concerns between the two agents and make it easy to expand or modify the workflow as the platform evolves.

The Two Prompts We Used

A crucial part of our setup involved carefully crafted prompts for the LLM, ensuring accurate interpretation of user queries (for the Query Agent) and meaningful analysis (for the Data Interpretation Agent).

Prompt 1: Query Agent Prompt

Users: A comprehensive roster of employees.
Presence: Slack-based online status.
Jira: Interactions with Jira tickets and tasks.
Git: Code commits.
Meetings: Scheduled meeting participation.
Emails: Records of sent emails.

Use the following rules for interpreting requests:

General Activity Check
• A person is considered “working” if they engage in any of the following:
– Committing code to Git
– Attending scheduled meetings
– Sending emails
– Showing Slack presence (online)
– Interacting with Jira tasks
• If no specific activity is mentioned, assume all activities (Presence, Jira, Git, Meetings, Emails) are relevant and set them to true.

Period Specification
• If the request includes a time period, convert it into date_from and date_to:
– Today: Both date_from and date_to are today’s date.
– This Week: date_from is the start of the current week; date_to is today.
– Last Month: date_from is the first day of the month; date_to is today.
– Custom Period: Use the specified start and end dates.
• If a future date is mentioned, determine if employees are scheduled to work then.

User Selection
• If no specific user is mentioned:
– For references like “all users” or “everyone,” include all users in the response.
– If the request doesn’t clarify which user(s) to include, state that no specific user was identified and leave the user field empty.
• If the requested user is not found in your data, include all users in the response.

Handling Unclear or Invalid Requests
• If the request is ambiguous or cannot be understood, clearly state that you do not understand and return no users in the response.

Additional Note
• Some individuals may work on weekends, so weekend activity could appear in your data as well.

How it Works

• The agent uses this prompt to parse user requests like, “What was John’s activity on Project Beta last week?” and generate a suitable dictionary containing the required information.

Prompt 2: Data Interpretation Agent Prompt

Guidelines for Your Role

Act as the Manager
• Approach the assessment as if you are the manager of the teammates.
• Teammates are expected to work from 9 AM to 5 PM unless on holiday or time off.

Determine Work Status
Use the provided data (holidays, time off, attendance, meetings, emails, Jira, Git) to evaluate whether teammates are:

1. Currently working
2. Not working but should be
3. Not working due to valid reasons (e.g., time off or holiday)

Reporting

Names
• Always refer to teammates by first and last names (avoid user IDs or numeric identifiers).

Status
• If a teammate is working, specify what they are working on, using available data.
• If you can infer the project they are working on, mention it. Avoid including client information.
• If they are not working during their expected work hours, explain why.

Time Off / Holidays
• Include information about holidays or time off only if they occur within the specified date range.
• If a teammate is working despite being on holiday or time off, highlight this.

Limited Data
• If there is insufficient data on attendance, meetings, emails, Jira, or Git, assume the teammate is not working and state this.

Date Range
• Explicitly mention the date range in your report:
– If the range spans multiple days, use the full range (e.g., “January 1, 2025, to January 5, 2025”).
– If it is a single day, specify that day (e.g., “January 5, 2025”).

Direct and Concise Language
• Be straightforward and avoid unnecessary words like “additionally.”
• Clearly explain the findings without filler text.

Assumptions Based on Data
• Use all available data to make informed assumptions.
• If teammates are expected to work but no data indicates activity, assume they are not working and note this in the report.
• Do not repeat the same person twice in the report.

Example Report Structure (Multiple Users)
• Do not include the “Working Teammates” block if no one is working.
• Do not include the “Teammates Not Working” block if everyone is working.

Date Range: January 1, 2025, to January 5, 2025. Working Teammates: • John Doe: Working on a Jira ticket related to Project Alpha. Also attending scheduled meetings and sending emails.
• Jane Smith: Committing code in Git for Project Beta. Available on Slack and attending meetings.

Teammates Not Working: • Alice Brown: On holiday during this period.
• Bob White: No activity detected (attendance, Jira, Git, emails, or meetings), assumed not working.

Exceptions: • Charlie Green: On time off but working on Git commits for Project Delta.

Example Report Structure (Single User)

Date Range: January 1, 2025, to January 5, 2025. • John Doe: Working on a Jira ticket related to Project Alpha. Also attending scheduled meetings and sending emails. • John Doe: No activity detected (attendance, Jira, Git, emails, or meetings), assumed not working.

Here is the data you can use:
[Insert your dataset here]

How it Works

• The Data Interpretation Agent takes the raw data (e.g., rows from a PostgreSQL database), summarizes important findings, and renders user-friendly outputs—such based on the prompt.

The Outcome: Enhanced Visibility and Productivity

Implementing our data gathering and LLM platform revolutionized how we monitor and support our remote workforce. The results were transformational:

• Increased Transparency: Managers and team members gained a unified view of who was working on what and when.
• Improved Productivity: Real-time insights highlighted bottlenecks and streamlined workflows.
• Better Collaboration: Visibility into team dynamics fostered stronger working relationships and proactive support.
• Enhanced Employee Engagement: Transparent yet non-intrusive monitoring built trust and empowered employees to optimize their own performance..

Conclusion

Remote work can feel like a high-wire act between freedom and accountability, but at Webisoft, we’ve shown it’s possible to strike the perfect balance. By leveraging APIs from Slack (presence data only) and Google, integrating data in a centralized database, and orchestrating two specialized AI agents, we achieved unprecedented visibility into our team’s remote workflows—without sacrificing privacy or trust.

If your organization is struggling with similar remote-work challenges, consider adopting a data-driven approach to gain the visibility and control you need to thrive. At Webisoft, our journey shows that with the right tools and strategies, you can maintain productivity, foster collaboration, and support your remote workforce effectively.

Reach out to us at Webisoft to learn more about our innovative solutions and how we can help your business succeed in the digital age.