project_bookworm/Local_CPU_LLM_Ollama_Mistral.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c3124e6f5d96e05f",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "# LangChain Ollama Mistral CPU\n",
    "\n",
    "* Ollama simplifies hosting LLMs and provides a simple API to interact with them.\n",
    " * https://ollama.com/ \n",
    "  * https://ollama.com/mistral\n",
    "* LangChain is a platform for building and deploying language models.\n",
    "  * https://python.langchain.com/ \n",
    "  * https://python.langchain.com/docs/integrations/llms/\n",
    "*   https://python.langchain.com/docs/integrations/llms/ollama \n",
    "* Mistral \n",
    "  * https://mistral.ai/\n",
    "  * https://mistral.ai/technology/#models\n",
    "  \n",
    "\n",
    "This playbook is CPU-only. \n",
    "Ollama runs Mistral in a Docker container with the default API endpoint\n",
    "\n",
    "```\n",
    "curl http://localhost:11434/api/generate -d '{\n",
    "  \"model\": \"mistral\",\n",
    "  \"prompt\":\"Why is the sky blue?\"\n",
    "}'\n",
    "```\n",
    "\n",
    "The LLM is hosted locally and needs no remote API.\n",
    "It's on-premises self-hosted and deployed on Linux (Ubuntu Server).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d566a43e6033d8d4",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "## Basic use with in-memory cache"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "initial_id",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-21T08:38:20.697371Z",
     "start_time": "2024-03-21T08:37:39.510775Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 748 ms, sys: 68.3 ms, total: 817 ms\n",
      "Wall time: 43.6 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" The reason why the sky appears blue during a clear day is due to a natural phenomenon called Rayleigh scattering. When sunlight, which is made up of different wavelengths or colors, collides with molecules and particles in Earth's atmosphere, shorter wavelengths like blue get scattered more easily than longer wavelengths like red. As a result, the sky appears blue during the day because our eyes are more sensitive to blue light and we see it more readily when looking at the sky.\\n\\nHowever, as the sun sets, the angle of the sunlight reaching Earth changes, causing the longer wavelengths like red and orange to be scattered in greater quantities. This is why we see beautiful red, pink, and orange hues during sunrise and sunset instead of blue.\""
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "from langchain_community.llms import Ollama\n",
    "from langchain.globals import set_llm_cache\n",
    "from langchain.cache import InMemoryCache\n",
    "\n",
    "set_llm_cache(InMemoryCache())\n",
    "\n",
    "llm = Ollama(model=\"mistral\")\n",
    "\n",
    "llm.invoke(\"Why is the sky blue?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "609a24f85a932ee6",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### Speed-up with in-mem cache"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "33a9510c0fbbfce2",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-21T08:47:29.318321Z",
     "start_time": "2024-03-21T08:47:29.313455Z"
    },
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 276 µs, sys: 0 ns, total: 276 µs\n",
      "Wall time: 279 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" The reason why the sky appears blue during a clear day is due to a natural phenomenon called Rayleigh scattering. When sunlight, which is made up of different wavelengths or colors, collides with molecules and particles in Earth's atmosphere, shorter wavelengths like blue get scattered more easily than longer wavelengths like red. As a result, the sky appears blue during the day because our eyes are more sensitive to blue light and we see it more readily when looking at the sky.\\n\\nHowever, as the sun sets, the angle of the sunlight reaching Earth changes, causing the longer wavelengths like red and orange to be scattered in greater quantities. This is why we see beautiful red, pink, and orange hues during sunrise and sunset instead of blue.\""
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "llm.invoke(\"Why is the sky blue?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aec810d53214e101",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}