How to Build an AI Agent That Reads and Summarizes Your Emails

How to Build an AI Agent That Reads and Summarizes Your Emails

Welcome back to AI Automation Guru. If there is one universal truth in the modern professional world, it is this: we all spend way too much time managing our inboxes. The average professional spends over a quarter of their workweek just reading and responding to emails. It is a massive drain on productivity.

But what if you could outsource this tedious task to a digital coworker? Today, we are going to build a functional, autonomous AI email agent. This agent will connect directly to your inbox, find your unread messages, read the contents, and use an advanced Large Language Model (LLM) to generate a concise, actionable summary of everything you need to know.

By the end of this comprehensive guide, you will have a working Python script that leverages the Gmail API and Google's Gemini API to completely automate your inbox triage.

The Architecture: How Our AI Email Agent Works

Before we dive into the code, let's understand the architecture of the agent we are building. An AI agent is essentially software that can perceive its environment, make decisions, and take actions. Here is our agent's workflow:

1. Perception (Sensors): The agent uses the Gmail API to securely connect to your Google account and scan for emails labeled "UNREAD."
2. Data Processing: It extracts the essential metadata (Sender, Subject) and parses the email body, stripping away messy HTML tags to get clean, readable text.
3. The Brain (LLM): The agent sends the clean text to the Gemini API (our reasoning engine) with a specific prompt instruction: "Summarize this email and highlight any action items."
4. Action (Actuators): Finally, the agent compiles these summaries into a clean report (or can be configured to send you a single digest email on Slack or Discord).

Prerequisites: Setting Up Your APIs

To build this, you need two keys: one to access your emails, and one to access the AI brain.

1. Get Your Gemini API Key

Google offers generous free tiers for the Gemini API. Head over to Google AI Studio, sign in with your Google account, and click "Create API Key." Save this key somewhere secure.

2. Enable the Gmail API

This is slightly more involved, but you only have to do it once:

• Go to the Google Cloud Console.
• Create a new project (e.g., "AI-Email-Agent").
• Navigate to APIs & Services > Library, search for "Gmail API," and click Enable.
• Go to Credentials, click "Create Credentials," and select OAuth client ID. (You may need to configure the OAuth consent screen first—just select "Desktop App" and set your email as a test user).
• Download the JSON file containing your credentials and save it in your project folder as credentials.json.

Step 1: Installing Required Python Libraries

Open your terminal or command prompt and install the necessary Python packages. We need the Google Client Library for Gmail, and the Generative AI library for Gemini.

pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
pip install google-generativeai
pip install beautifulsoup4
  

Note: We use BeautifulSoup to easily strip HTML tags out of the email bodies before sending them to the AI, saving tokens and improving accuracy.

Step 2: Authenticating and Fetching Emails

First, let's write the code to securely log into your Gmail and fetch the raw data. Create a new file called email_agent.py.

import os.path
import base64
from bs4 import BeautifulSoup
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build

# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']

def authenticate_gmail():
    """Shows basic usage of the Gmail API.
    Lists the user's Gmail labels.
    """
    creds = None
    # The file token.json stores the user's access and refresh tokens.
    if os.path.exists('token.json'):
        creds = Credentials.from_authorized_user_file('token.json', SCOPES)
    
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file('credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.json', 'w') as token:
            token.write(creds.to_json())

    return build('gmail', 'v1', credentials=creds)

def clean_email_body(html_body):
    """Removes HTML tags to return plain text."""
    soup = BeautifulSoup(html_body, "html.parser")
    return soup.get_text(separator="\n", strip=True)
  

When you run this for the first time, a browser window will open asking you to grant permission for the app to view your emails. Once granted, it saves a token.json file so you do not have to log in every time.

Step 3: Integrating the Gemini AI Brain

Now that we can connect to Gmail, we need to set up the summarization engine. We will use the google-generativeai SDK. Add this to your email_agent.py file:

import google.generativeai as genai

# Replace with your actual API key, or better yet, load it from an environment variable
GENAI_API_KEY = "YOUR_GEMINI_API_KEY" 
genai.configure(api_key=GENAI_API_KEY)

# Initialize the model (Gemini 1.5 Flash is incredibly fast and perfect for text tasks)
model = genai.GenerativeModel('gemini-1.5-flash')

def summarize_email(sender, subject, body):
    """Sends the email content to Gemini for summarization."""
    prompt = f"""
    You are a highly efficient executive assistant AI. 
    Read the following email and provide a brief, 2-3 sentence summary.
    If there are any explicit action items or deadlines for me, list them in bullet points.
    
    Sender: {sender}
    Subject: {subject}
    Email Body: {body}
    """
    
    try:
        response = model.generate_content(prompt)
        return response.text
    except Exception as e:
        return f"Error summarizing email: {e}"
  

Step 4: Putting It All Together (The Core Execution Loop)

We have our sensors (Gmail API) and our brain (Gemini). Now, let's write the execution loop that brings the agent to life by fetching unread messages, parsing the data, and printing the summaries.

def main():
    service = authenticate_gmail()
    print("Agent Authenticated. Scanning inbox...")

    # Fetch up to 5 unread emails from the inbox
    results = service.users().messages().list(userId='me', labelIds=['INBOX', 'UNREAD'], maxResults=5).execute()
    messages = results.get('messages', [])

    if not messages:
        print('Inbox zero! No new unread messages.')
        return

    print(f"Found {len(messages)} unread messages. Processing...\n")
    print("-" * 50)

    for msg in messages:
        # Get the full email data
        message_data = service.users().messages().get(userId='me', id=msg['id'], format='full').execute()
        
        headers = message_data['payload']['headers']
        subject = next((header['value'] for header in headers if header['name'] == 'Subject'), 'No Subject')
        sender = next((header['value'] for header in headers if header['name'] == 'From'), 'Unknown Sender')

        # Extracting the email body (handling multi-part emails)
        body_data = ""
        if 'parts' in message_data['payload']:
            for part in message_data['payload']['parts']:
                if part['mimeType'] == 'text/plain':
                    body_data = part['body'].get('data', '')
                    break
                elif part['mimeType'] == 'text/html':
                    body_data = part['body'].get('data', '')
        else:
            body_data = message_data['payload']['body'].get('data', '')

        if body_data:
            # Decode the base64 encoded email body
            decoded_body = base64.urlsafe_b64decode(body_data).decode('utf-8')
            clean_body = clean_email_body(decoded_body)
            
            print(f"Analyzing Email from: {sender}")
            print(f"Subject: {subject}")
            
            # Pass to Gemini for summarization
            summary = summarize_email(sender, subject, clean_body)
            
            print("\n--- AI SUMMARY ---")
            print(summary)
            print("-" * 50)

if __name__ == '__main__':
    main()
  

How to Scale Your Agentic Workflow

Congratulations! You just built a functional AI agent. Right now, it runs when you execute the script. However, true automation means it runs while you sleep. Here is how you can take this to the next level:

• Cron Jobs (Mac/Linux) or Task Scheduler (Windows): Set this script to run automatically every morning at 8:00 AM, so you wake up to a clean digest.
• Webhook Integration: Instead of printing the summaries to your terminal, you can add a simple requests.post() function to send the generated AI summaries directly to a private Discord or Slack channel.
• Auto-Drafting Replies: You can expand the Gemini prompt to not only summarize the email but also generate a polite draft reply. You could then use the Gmail API's messages().send() endpoint to save that draft directly into your Gmail account, ready for your approval.

Conclusion

The beauty of building your own AI agents using Python is that you are not locked into the constraints of expensive SaaS products. You have total control over the logic, the LLM prompt, and the data privacy. By combining the data retrieval power of the Gmail API with the reasoning capabilities of Gemini, you have created a digital assistant that can save you hundreds of hours a year.

Let me know in the comments how you plan to customize this agent! Will you have it filter out spam, categorize invoices, or draft your replies? The possibilities are endless.