10 Jan 2024 · Software Engineering

    Gemini Pro vs ChatGPT and Bard: The Search for the Best LLM

    3 min read

    Gemini Pro: Exploring Google’s New Large Language Model

    Google recently unveiled Gemini Pro, its latest AI language model, and the tech community is buzzing. Gemini, a large language model, is designed to operate in three distinct environments: smartphones, as a direct competitor or replacement for chatbots like ChatGPT, and in its most advanced form, the ‘ultra’ version.

    Getting Started with Gemini on Google Cloud

    For those keen to test Gemini’s skills, Google Cloud’s Vertex AI service is the gateway. Unlike the more straightforward interface of ChatGPT, Vertex AI demands a certain familiarity with Google’s cloud ecosystem.

    To begin, you need a Google account to create a project and set up a billing account. Adding a credit card is necessary, but charges apply only for certain services, with a free tier available for basic usage.

    Once you navigate through the initial setup, Gemini is accessible through Vertex AI. The service’s ‘model garden’ hosts various AI models, including Gemini, which users can deploy and train directly in Google Cloud.

    The model garden on Google Cloud. You can run any of these proprietary or open-source models here.

    Testing Gemini’s Capabilities: A Mixed Bag of Results

    My testing experience with Gemini shower a mix of hits and misses. For instance, in a logical reasoning test using a riddle, Gemini fell short, aligning more with an earlier version of ChatGPT-3.5 rather than its latest ChatGPT-4. However, its multimodal version, capable of processing images, fared much better. It accurately recognized objects in drawings and deciphered handwritten text, showcasing a performance on par with ChatGPT-4 in similar tasks.

    Gemini’s coding assistance, however, was disappointing. Tasked with the Traveling Salesperson Problem, a classic computational challenge, Gemini struggled to produce accurate, functional code on the first shot. In previous testing we found ChatGPT-4 and Copilot to be give the best coding assistance overall.

    ReasoningImage recognitionCoding
    Gemini Pro
    FailGemini Pro
    Gemini Pro

    The Potential of Gemini’s API

    One of the more exciting features of Gemini is its API, which allows developers to integrate the model into their applications. Setting up involves authenticating with Google Cloud, a process that, once completed, offers a wide range of development possibilities. The API notably provides detailed responses, including safety ratings for outputs, a unique and valuable feature for developers.

    Gemini’s API helper. You can get the code to reproduce any conversation using the API.

    Conclusion: A Step Forward with Room for Growth

    Gemini, with its impressive image processing abilities and unique safety features, certainly marks a step forward in AI technology. However, its performance in tasks like coding assistance leaves room for improvement. The tech community eagerly awaits the release of the ultra model, which promises to surpass current AI models, including ChatGPT.

    As Google continues to refine Gemini, the free tier of Google Cloud’s Vertex AI offers a fantastic opportunity for developers and tech enthusiasts to experiment and contribute to this evolving AI landscape. Future explorations within Vertex AI, such as training models and preparing datasets for fine-tuning, are avenues ripe for exploration. Stay tuned for more updates and insights in the world of AI.

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Writen by:
    I picked up most of my skills during the years I worked at IBM. Was a DBA, developer, and cloud engineer for a time. After that, I went into freelancing, where I found the passion for writing. Now, I'm a full-time writer at Semaphore.