Have you ever wondered what it would be like to chat with a perpetually gloomy AI? Meet Marvin, a sarcastic chatbot inspired by Douglas Adams’ iconic character from “The Hitchhiker’s Guide to the Galaxy.” But Marvin is more than just a witty conversationalist – he’s a testament to the incredible ease and flexibility of deploying Large Language Models (LLMs) using Cloudflare Workers. Cloudflare’s free tier opens up a lot of possibilities for campaigners to think more about how to leverage LLM in their work and engage their audiences.
How Marvin Was Made
Creating Marvin was surprisingly straightforward, thanks to the power of Cloudflare Workers and their AI capabilities. Here’s a brief overview of the process:
- Cloudflare Workers: Providing serverless computing.
- AI Gateway: Used to manage and control AI requests, offering analytics, and improved performance.
- Mistral 7B Instruct Model: The “brain” behind Marvin’s responses, fine-tuned to provide a sarcastic and gloomy tone. I used Mistral as it’s easy to fine tune, and one of the more ethical companies out there.
- HTML/JavaScript Frontend: A simple, responsive interface for users to interact with Marvin.
Ultimately, this project was possible due to the Cloudflare Workers tools. It’s a powerful platform that allows developers to run various AI models directly on Cloudflare’s global network. And it supports multiple Large Language Models (LLMs) from different providers, including OpenAI, Anthropic, Meta, and Mistral, among others. This means developers can choose from a wide range of pre-trained models for tasks like text generation, summarisation, or code completion.
This flexibility means that Marvin could easily be adapted to use different models or even combine multiple models for more complex tasks.
While Marvin started as a bit of personal fun, he represents something much more significant. He demonstrates how quickly, easily, and cost-effectively LLMs can be deployed in a cloud environment. But also how much work is needed to make them safe. Of the 10,382 characters in this code; 2,182 were dedicated to the system prompt and in particular telling it what not to do, especially around illegal or unethical Issues. Before this fine-tuning, it had almost no guardrails.
The simplicity of deploying AIs like Marvin opens up a world of possibilities for campaigners and non-profit organisations. Here are just a few ideas:
- Code Development: Use AI to assist in writing and debugging code for campaign websites, apps, or extending CRM functions.
- Data Analysis: Leverage AI to analyse campaign data and extract meaningful insights. (But perhaps be conscious of data privacy issues. Although owning the LLM doing the work might be a bit better)
- Personalised User Interactions: Create chatbots that can provide tailored information to supporters or service users.
- Personally, I think this is where I see LLMs being useful in the immediate future, helping service provides give information in a more intuitive fashion.
There are some great AI tools out there for campaigners. From systems set to help write emails to predictive analytics. They’re all very early days and need a lot more refining, but the potential is pretty much limitless.
As LLMs and the platforms that host them continue to develop at pace, the barriers to building AI applications has never been lower.
Next project might be something a bit more useful and practical!