marius
/
project_bookworm
mirror of https://github.com/norandom/project_bookworm
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

LangChain, Mistral, Ollama, local LLM CPU-hosted

main
Marius Ciepluch 2024-03-21 08:54:28 +00:00
parent ad0acff4ac
commit bfba2c8342
1 changed files with 36 additions and 21 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
Local_CPU_LLM_Ollama_Mistral.ipynb
View File

@ -4,7 +4,10 @@
"cell_type": "markdown",
"id": "c3124e6f5d96e05f",
"metadata": {
"collapsed": false
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"# LangChain Ollama Mistral CPU\n",
@ -39,7 +42,10 @@
"cell_type": "markdown",
"id": "d566a43e6033d8d4",
"metadata": {
"collapsed": false
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"## Basic use with in-memory cache"
@ -47,31 +53,30 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 1,
"id": "initial_id",
"metadata": {
"ExecuteTime": {
"end_time": "2024-03-21T08:38:20.697371Z",
"start_time": "2024-03-21T08:37:39.510775Z"
},
"collapsed": true
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 30.5 ms, sys: 4.16 ms, total: 34.6 ms\n",
"Wall time: 41.2 s\n"
"CPU times: user 748 ms, sys: 68.3 ms, total: 817 ms\n",
"Wall time: 43.6 s\n"
]
},
{
"data": {
"text/plain": [
"\" The reason why the sky appears blue during a clear day is due to a particular type of scattering called Rayleigh scattering. When the sun's rays enter Earth's atmosphere, they are made up of various wavelengths or colors of light. Short-wavelength light, such as blue and violet, gets scattered more easily because they have smaller wavelengths. This scattered blue light is what we see when we look at the sky. However, our eyes are more sensitive to blue light and less sensitive to violet light, making the sky appear more blue than violet. At sunrise and sunset, the sky can take on a variety of colors because the angles of the sunlight entering the atmosphere cause different types of scattering to occur.\""
"\" The reason why the sky appears blue during a clear day is due to a natural phenomenon called Rayleigh scattering. When sunlight, which is made up of different wavelengths or colors, collides with molecules and particles in Earth's atmosphere, shorter wavelengths like blue get scattered more easily than longer wavelengths like red. As a result, the sky appears blue during the day because our eyes are more sensitive to blue light and we see it more readily when looking at the sky.\\n\\nHowever, as the sun sets, the angle of the sunlight reaching Earth changes, causing the longer wavelengths like red and orange to be scattered in greater quantities. This is why we see beautiful red, pink, and orange hues during sunrise and sunset instead of blue.\""
]
},
"execution_count": 10,
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
@ -93,7 +98,10 @@
"cell_type": "markdown",
"id": "609a24f85a932ee6",
"metadata": {
"collapsed": false
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"### Speed-up with in-mem cache"
@ -101,31 +109,34 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 2,
"id": "33a9510c0fbbfce2",
"metadata": {
"ExecuteTime": {
"end_time": "2024-03-21T08:47:29.318321Z",
"start_time": "2024-03-21T08:47:29.313455Z"
},
"collapsed": false
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 0 ns, sys: 276 µs, total: 276 µs\n",
"Wall time: 282 µs\n"
"CPU times: user 276 µs, sys: 0 ns, total: 276 µs\n",
"Wall time: 279 µs\n"
]
},
{
"data": {
"text/plain": [
"\" The reason why the sky appears blue during a clear day is due to a particular type of scattering called Rayleigh scattering. When the sun's rays enter Earth's atmosphere, they are made up of various wavelengths or colors of light. Short-wavelength light, such as blue and violet, gets scattered more easily because they have smaller wavelengths. This scattered blue light is what we see when we look at the sky. However, our eyes are more sensitive to blue light and less sensitive to violet light, making the sky appear more blue than violet. At sunrise and sunset, the sky can take on a variety of colors because the angles of the sunlight entering the atmosphere cause different types of scattering to occur.\""
"\" The reason why the sky appears blue during a clear day is due to a natural phenomenon called Rayleigh scattering. When sunlight, which is made up of different wavelengths or colors, collides with molecules and particles in Earth's atmosphere, shorter wavelengths like blue get scattered more easily than longer wavelengths like red. As a result, the sky appears blue during the day because our eyes are more sensitive to blue light and we see it more readily when looking at the sky.\\n\\nHowever, as the sun sets, the angle of the sunlight reaching Earth changes, causing the longer wavelengths like red and orange to be scattered in greater quantities. This is why we see beautiful red, pink, and orange hues during sunrise and sunset instead of blue.\""
]
},
"execution_count": 11,
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
@ -137,9 +148,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"id": "aec810d53214e101",
"metadata": {
"collapsed": false
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": []
@ -147,21 +162,21 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
"pygments_lexer": "ipython3",
"version": "3.11.8"
}
},
"nbformat": 4,