marius/project_bookworm
1
0
Fork 0
mirror of https://github.com/norandom/project_bookworm.git synced 2024-12-04 19:13:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: marius:0118544c7dda1ef66befa487ceabda9fdbdd74fb
Branches Tags
marius:main
..
compare: marius:30ebb519a1c6f7dc5fbe61f50c2ac7981ab1de06
Branches Tags
marius:main

5 Commits
0118544c7d ... 30ebb519a1

Author SHA1 Message Date
Marius Ciepluch
30ebb519a1 Created using Colaboratory 2024-04-04 14:38:48 +02:00
Marius Ciepluch
c422d14692 updated requirements txt to use faiss cpu 2024-04-04 11:22:38 +00:00
Marius Ciepluch
6fd67189d1 Created using Colaboratory 2024-04-04 13:20:46 +02:00
Marius Ciepluch
2d7e889968 requirements gpu txt 2024-04-04 10:47:54 +00:00
Marius Ciepluch
e56c87a71d moved from sqlite to faiss for concurrency reasons 2024-04-04 10:38:12 +00:00
6 changed files with 1666 additions and 953 deletions
Show Stats Expand all files Collapse all files

1363
EverNote_Documents_To_FAISS_Colab_GPU.ipynb Normal file
View File

File diff suppressed because it is too large Load Diff

1229
EverNote_To_OpenAI.ipynb
View File

File diff suppressed because one or more lines are too long

0
export.documents.txt → data/export.documents.txt
View File

0
export.txt → data/export.txt
View File

23
requirements.gpu.txt Normal file
View File

@ -0,0 +1,23 @@
openai==1.13.3
langchain==0.1.10
pinecone-client==3.1.0
python-dotenv==1.0.1
tiktoken==0.6.0
wikipedia==1.4.0
pypdf[full]==4.0.2
langchain_openai==0.0.8
langchain_experimental==0.0.53
langchainhub==0.1.14
ipywidgets
cryptography==42.0.5
pikepdf==8.13.0
html2text==2024.2.26
transformers==4.39.0
torch
torchvision
torchaudio
sentence-transformers==2.5.1
sqlite-vss==0.1.2
faiss_cpu==1.8.0

4
requirements.txt
View File

@ -8,14 +8,16 @@ pypdf[full]==4.0.2
langchain_openai==0.0.8 langchain_openai==0.0.8
langchain_experimental==0.0.53 langchain_experimental==0.0.53
langchainhub==0.1.14 langchainhub==0.1.14
ipywidgets==8.1.2 ipywidgets
cryptography==42.0.5 cryptography==42.0.5
pikepdf==8.13.0 pikepdf==8.13.0
html2text==2024.2.26
transformers==4.39.0 transformers==4.39.0
torch==2.2.1+cpu torch==2.2.1+cpu
torchvision==0.17.1+cpu torchvision==0.17.1+cpu
torchaudio==2.2.1+cpu torchaudio==2.2.1+cpu
sentence-transformers==2.5.1 sentence-transformers==2.5.1
sqlite-vss==0.1.2 sqlite-vss==0.1.2
faiss_cpu==1.8.0