marius/trading_analysis
1
0
Fork 0
mirror of https://github.com/norandom/trading_analysis.git synced 2024-11-21 20:53:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
67dc11ace4
trading_analysis/Silver_Fundamentals.ipynb

1850 lines
165 KiB
Plaintext
Raw Normal View History

Silver analysis
2024-08-19 17:41:20 +00:00
{
"cells": [
{
"cell_type": "markdown",
"id": "bf16a6eb-44a2-4dd7-8d4d-598e6f2c9b24",
"metadata": {},
"source": [
"# Silver "
]
},
{
"cell_type": "markdown",
"id": "4405f172-43b6-4245-bcd8-2c6e01fada3a",
"metadata": {},
"source": [
"We can download the data from the CFTC via Python.\n",
"\n",
"Many data APIs become commercial for recent prices. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f3704b41-1622-4596-ae99-3201b9bd8e14",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Selected: disaggregated_fut\n",
"Downloaded single year data from: 2020\n",
"Stored the file f_year.txt in the working directory.\n",
"Selected: disaggregated_fut\n",
"Downloaded single year data from: 2021\n",
"Stored the file f_year.txt in the working directory.\n",
"Selected: disaggregated_fut\n",
"Downloaded single year data from: 2022\n",
"Stored the file f_year.txt in the working directory.\n",
"Selected: disaggregated_fut\n",
"Downloaded single year data from: 2023\n",
"Stored the file f_year.txt in the working directory.\n",
"Selected: disaggregated_fut\n",
"Downloaded single year data from: 2024\n",
"Stored the file f_year.txt in the working directory.\n"
]
}
],
"source": [
"import pandas as pd\n",
"import cot_reports as cot # Ensure cot_reports is correctly imported and cot.cot_year() works as expected\n",
"\n",
"df = pd.DataFrame()\n",
"begin_year = 2020\n",
"end_year = 2024\n",
"\n",
"for i in range(begin_year, end_year + 1):\n",
" # Assuming cot.cot_year returns a DataFrame\n",
" single_year = pd.DataFrame(cot.cot_year(i, cot_report_type='disaggregated_fut'))\n",
" single_year.to_csv(f'./COT_CFTC_{i}.csv', index=False)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f4f90eb0-dca4-4589-a385-408c66c6e0a0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['COT_CFTC_2022.csv', 'COT_CFTC_2023.csv', 'COT_CFTC_2021.csv', 'COT_CFTC_2020.csv', 'COT_CFTC_2024.csv']\n"
]
}
],
"source": [
"import glob\n",
"\n",
"# Adjust the path and pattern according to your CSV files location and naming convention\n",
"csv_files = glob.glob('COT_CFTC_20*.csv')\n",
"print(csv_files)"
]
},
{
"cell_type": "markdown",
"id": "414b7248-5acc-46a9-916a-cbed7228331f",
"metadata": {},
"source": [
"At this point, we have the CFTC Disaggregated Futures report from 2022 to 2024 (until today)"
]
},
{
"cell_type": "markdown",
"id": "084eb98b-1914-49e0-b57a-4e7717ea159c",
"metadata": {},
"source": [
"## Load the data, and prepare the dataset\n",
"\n",
"The CFTC report contains data on all futures, not just silver."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "cb167036-4f9b-4405-a942-71fae7d7aec7",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_1629470/490941526.py:1: DtypeWarning: Columns (120,121,132,133,134,135,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160) have mixed types. Specify dtype option on import or set low_memory=False.\n",
" df = pd.read_csv('COT_CFTC_2024.csv')\n"
]
}
],
"source": [
"df = pd.read_csv('COT_CFTC_2024.csv')"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2f554d5a-e6da-4a6c-a16e-f2b3294b4824",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(8344, 191)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"df[\"Report_Date_as_YYYY-MM-DD\"] = pd.to_datetime(df[\"Report_Date_as_YYYY-MM-DD\"], format='%Y-%m-%d')\n",
"df.set_index(\"Report_Date_as_YYYY-MM-DD\")\n",
"df.shape"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "4bd367af-8385-406a-87ee-5aa68b3a8b7a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"8345\n"
]
}
],
"source": [
"!cat COT_CFTC_2024.csv | wc -l"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "fc275bed-93a3-4c66-87e0-7312aa712f6a",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Market_and_Exchange_Names</th>\n",
" <th>As_of_Date_In_Form_YYMMDD</th>\n",
" <th>Report_Date_as_YYYY-MM-DD</th>\n",
" <th>CFTC_Contract_Market_Code</th>\n",
" <th>CFTC_Market_Code</th>\n",
" <th>CFTC_Region_Code</th>\n",
" <th>CFTC_Commodity_Code</th>\n",
" <th>Open_Interest_All</th>\n",
" <th>Prod_Merc_Positions_Long_All</th>\n",
" <th>Prod_Merc_Positions_Short_All</th>\n",
" <th>...</th>\n",
" <th>Conc_Net_LE_4_TDR_Long_Other</th>\n",
" <th>Conc_Net_LE_4_TDR_Short_Other</th>\n",
" <th>Conc_Net_LE_8_TDR_Long_Other</th>\n",
" <th>Conc_Net_LE_8_TDR_Short_Other</th>\n",
" <th>Contract_Units</th>\n",
" <th>CFTC_Contract_Market_Code_Quotes</th>\n",
" <th>CFTC_Market_Code_Quotes</th>\n",
" <th>CFTC_Commodity_Code_Quotes</th>\n",
" <th>CFTC_SubGroup_Code</th>\n",
" <th>FutOnly_or_Combined</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>WHEAT-SRW - CHICAGO BOARD OF TRADE</td>\n",
" <td>240813</td>\n",
" <td>2024-08-13</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>401604</td>\n",
" <td>68899</td>\n",
" <td>85867</td>\n",
" <td>...</td>\n",
" <td>39.0</td>\n",
" <td>31.7</td>\n",
" <td>54.2</td>\n",
" <td>42.6</td>\n",
" <td>(CONTRACTS OF 5,000 BUSHELS)</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>1</td>\n",
" <td>A10</td>\n",
" <td>FutOnly</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>WHEAT-SRW - CHICAGO BOARD OF TRADE</td>\n",
" <td>240806</td>\n",
" <td>2024-08-06</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>431547</td>\n",
" <td>73401</td>\n",
" <td>85750</td>\n",
" <td>...</td>\n",
" <td>41.6</td>\n",
" <td>32.3</td>\n",
" <td>57.3</td>\n",
" <td>41.7</td>\n",
" <td>(CONTRACTS OF 5,000 BUSHELS)</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>1</td>\n",
" <td>A10</td>\n",
" <td>FutOnly</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>WHEAT-SRW - CHICAGO BOARD OF TRADE</td>\n",
" <td>240730</td>\n",
" <td>2024-07-30</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>432863</td>\n",
" <td>73086</td>\n",
" <td>83875</td>\n",
" <td>...</td>\n",
" <td>46.0</td>\n",
" <td>28.5</td>\n",
" <td>60.8</td>\n",
" <td>38.4</td>\n",
" <td>(CONTRACTS OF 5,000 BUSHELS)</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>1</td>\n",
" <td>A10</td>\n",
" <td>FutOnly</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>WHEAT-SRW - CHICAGO BOARD OF TRADE</td>\n",
" <td>240723</td>\n",
" <td>2024-07-23</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>421227</td>\n",
" <td>63636</td>\n",
" <td>81374</td>\n",
" <td>...</td>\n",
" <td>48.3</td>\n",
" <td>27.9</td>\n",
" <td>63.5</td>\n",
" <td>37.7</td>\n",
" <td>(CONTRACTS OF 5,000 BUSHELS)</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>1</td>\n",
" <td>A10</td>\n",
" <td>FutOnly</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>WHEAT-SRW - CHICAGO BOARD OF TRADE</td>\n",
" <td>240716</td>\n",
" <td>2024-07-16</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>413885</td>\n",
" <td>64297</td>\n",
" <td>81558</td>\n",
" <td>...</td>\n",
" <td>50.4</td>\n",
" <td>28.0</td>\n",
" <td>65.9</td>\n",
" <td>37.3</td>\n",
" <td>(CONTRACTS OF 5,000 BUSHELS)</td>\n",
" <td>001602</td>\n",
" <td>CBT</td>\n",
" <td>1</td>\n",
" <td>A10</td>\n",
" <td>FutOnly</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>5 rows × 191 columns</p>\n",
"</div>"
],
"text/plain": [
" Market_and_Exchange_Names As_of_Date_In_Form_YYMMDD \\\n",
"0 WHEAT-SRW - CHICAGO BOARD OF TRADE 240813 \n",
"1 WHEAT-SRW - CHICAGO BOARD OF TRADE 240806 \n",
"2 WHEAT-SRW - CHICAGO BOARD OF TRADE 240730 \n",
"3 WHEAT-SRW - CHICAGO BOARD OF TRADE 240723 \n",
"4 WHEAT-SRW - CHICAGO BOARD OF TRADE 240716 \n",
"\n",
" Report_Date_as_YYYY-MM-DD CFTC_Contract_Market_Code CFTC_Market_Code \\\n",
"0 2024-08-13 001602 CBT \n",
"1 2024-08-06 001602 CBT \n",
"2 2024-07-30 001602 CBT \n",
"3 2024-07-23 001602 CBT \n",
"4 2024-07-16 001602 CBT \n",
"\n",
" CFTC_Region_Code CFTC_Commodity_Code Open_Interest_All \\\n",
"0 0 1 401604 \n",
"1 0 1 431547 \n",
"2 0 1 432863 \n",
"3 0 1 421227 \n",
"4 0 1 413885 \n",
"\n",
" Prod_Merc_Positions_Long_All Prod_Merc_Positions_Short_All ... \\\n",
"0 68899 85867 ... \n",
"1 73401 85750 ... \n",
"2 73086 83875 ... \n",
"3 63636 81374 ... \n",
"4 64297 81558 ... \n",
"\n",
" Conc_Net_LE_4_TDR_Long_Other Conc_Net_LE_4_TDR_Short_Other \\\n",
"0 39.0 31.7 \n",
"1 41.6 32.3 \n",
"2 46.0 28.5 \n",
"3 48.3 27.9 \n",
"4 50.4 28.0 \n",
"\n",
" Conc_Net_LE_8_TDR_Long_Other Conc_Net_LE_8_TDR_Short_Other \\\n",
"0 54.2 42.6 \n",
"1 57.3 41.7 \n",
"2 60.8 38.4 \n",
"3 63.5 37.7 \n",
"4 65.9 37.3 \n",
"\n",
" Contract_Units CFTC_Contract_Market_Code_Quotes \\\n",
"0 (CONTRACTS OF 5,000 BUSHELS) 001602 \n",
"1 (CONTRACTS OF 5,000 BUSHELS) 001602 \n",
"2 (CONTRACTS OF 5,000 BUSHELS) 001602 \n",
"3 (CONTRACTS OF 5,000 BUSHELS) 001602 \n",
"4 (CONTRACTS OF 5,000 BUSHELS) 001602 \n",
"\n",
" CFTC_Market_Code_Quotes CFTC_Commodity_Code_Quotes CFTC_SubGroup_Code \\\n",
"0 CBT 1 A10 \n",
"1 CBT 1 A10 \n",
"2 CBT 1 A10 \n",
"3 CBT 1 A10 \n",
"4 CBT 1 A10 \n",
"\n",
" FutOnly_or_Combined \n",
"0 FutOnly \n",
"1 FutOnly \n",
"2 FutOnly \n",
"3 FutOnly \n",
"4 FutOnly \n",
"\n",
"[5 rows x 191 columns]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "38311a58-d12d-49ae-9abb-a17bcc7f7a22",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"# Filter for rows where \"Market_and_Exchange_Names\" matches either of the specified values\n",
"silver_df = df[df[\"Market_and_Exchange_Names\"].isin([\"SILVER - COMMODITY EXCHANGE INC.\"])].copy()\n",
"\n",
"\n",
"# After filtering, you can standardize the \"Market_and_Exchange_Names\" if you want all of them to have the same name\n",
"# This is optional and based on your specific requirement to 'merge' under a unified label\n",
"silver_df[\"Market_and_Exchange_Names\"] = \"COPPER-GRADE #1 - COMMODITY EXCHANGE INC.\"\n",
"\n",
"silver_df[\"Report_Date_as_YYYY-MM-DD\"] = pd.to_datetime(df[\"Report_Date_as_YYYY-MM-DD\"], format='%Y-%m-%d')\n",
"silver_df = silver_df.set_index(\"Report_Date_as_YYYY-MM-DD\")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "af7ce715-7958-48bd-bb4e-2dbb4ec50171",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(33, 190)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"silver_df.shape # we have calendar week 33"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "0c1b24aa-db20-4935-828b-c7ca9f4959e4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['Market_and_Exchange_Names', 'As_of_Date_In_Form_YYMMDD', 'CFTC_Contract_Market_Code', 'CFTC_Market_Code', 'CFTC_Region_Code', 'CFTC_Commodity_Code', 'Open_Interest_All', 'Prod_Merc_Positions_Long_All', 'Prod_Merc_Positions_Short_All', 'Swap_Positions_Long_All', 'Swap__Positions_Short_All', 'Swap__Positions_Spread_All', 'M_Money_Positions_Long_All', 'M_Money_Positions_Short_All', 'M_Money_Positions_Spread_All', 'Other_Rept_Positions_Long_All', 'Other_Rept_Positions_Short_All', 'Other_Rept_Positions_Spread_All', 'Tot_Rept_Positions_Long_All', 'Tot_Rept_Positions_Short_All', 'NonRept_Positions_Long_All', 'NonRept_Positions_Short_All', 'Open_Interest_Old', 'Prod_Merc_Positions_Long_Old', 'Prod_Merc_Positions_Short_Old', 'Swap_Positions_Long_Old', 'Swap__Positions_Short_Old', 'Swap__Positions_Spread_Old', 'M_Money_Positions_Long_Old', 'M_Money_Positions_Short_Old', 'M_Money_Positions_Spread_Old', 'Other_Rept_Positions_Long_Old', 'Other_Rept_Positions_Short_Old', 'Other_Rept_Positions_Spread_Old', 'Tot_Rept_Positions_Long_Old', 'Tot_Rept_Positions_Short_Old', 'NonRept_Positions_Long_Old', 'NonRept_Positions_Short_Old', 'Open_Interest_Other', 'Prod_Merc_Positions_Long_Other', 'Prod_Merc_Positions_Short_Other', 'Swap_Positions_Long_Other', 'Swap__Positions_Short_Other', 'Swap__Positions_Spread_Other', 'M_Money_Positions_Long_Other', 'M_Money_Positions_Short_Other', 'M_Money_Positions_Spread_Other', 'Other_Rept_Positions_Long_Other', 'Other_Rept_Positions_Short_Other', 'Other_Rept_Positions_Spread_Other', 'Tot_Rept_Positions_Long_Other', 'Tot_Rept_Positions_Short_Other', 'NonRept_Positions_Long_Other', 'NonRept_Positions_Short_Other', 'Change_in_Open_Interest_All', 'Change_in_Prod_Merc_Long_All', 'Change_in_Prod_Merc_Short_All', 'Change_in_Swap_Long_All', 'Change_in_Swap_Short_All', 'Change_in_Swap_Spread_All', 'Change_in_M_Money_Long_All', 'Change_in_M_Money_Short_All', 'Change_in_M_Money_Spread_All', 'Change_in_Other_Rept_Long_All', 'Change_in_Other_Rept_Short_All', 'Change_in_Other_Rept_Spread_All', 'Change_in_Tot_Rept_Long_All', 'Change_in_Tot_Rept_Short_All', 'Change_in_NonRept_Long_All', 'Change_in_NonRept_Short_All', 'Pct_of_Open_Interest_All', 'Pct_of_OI_Prod_Merc_Long_All', 'Pct_of_OI_Prod_Merc_Short_All', 'Pct_of_OI_Swap_Long_All', 'Pct_of_OI_Swap_Short_All', 'Pct_of_OI_Swap_Spread_All', 'Pct_of_OI_M_Money_Long_All', 'Pct_of_OI_M_Money_Short_All', 'Pct_of_OI_M_Money_Spread_All', 'Pct_of_OI_Other_Rept_Long_All', 'Pct_of_OI_Other_Rept_Short_All', 'Pct_of_OI_Other_Rept_Spread_All', 'Pct_of_OI_Tot_Rept_Long_All', 'Pct_of_OI_Tot_Rept_Short_All', 'Pct_of_OI_NonRept_Long_All', 'Pct_of_OI_NonRept_Short_All', 'Pct_of_Open_Interest_Old', 'Pct_of_OI_Prod_Merc_Long_Old', 'Pct_of_OI_Prod_Merc_Short_Old', 'Pct_of_OI_Swap_Long_Old', 'Pct_of_OI_Swap_Short_Old', 'Pct_of_OI_Swap_Spread_Old', 'Pct_of_OI_M_Money_Long_Old', 'Pct_of_OI_M_Money_Short_Old', 'Pct_of_OI_M_Money_Spread_Old', 'Pct_of_OI_Other_Rept_Long_Old', 'Pct_of_OI_Other_Rept_Short_Old', 'Pct_of_OI_Other_Rept_Spread_Old', 'Pct_of_OI_Tot_Rept_Long_Old', 'Pct_of_OI_Tot_Rept_Short_Old', 'Pct_of_OI_NonRept_Long_Old', 'Pct_of_OI_NonRept_Short_Old', 'Pct_of_Open_Interest_Other', 'Pct_of_OI_Prod_Merc_Long_Other', 'Pct_of_OI_Prod_Merc_Short_Other', 'Pct_of_OI_Swap_Long_Other', 'Pct_of_OI_Swap_Short_Other', 'Pct_of_OI_Swap_Spread_Other', 'Pct_of_OI_M_Money_Long_Other', 'Pct_of_OI_M_Money_Short_Other', 'Pct_of_OI_M_Money_Spread_Other', 'Pct_of_OI_Other_Rept_Long_Other', 'Pct_of_OI_Other_Rept_Short_Other', 'Pct_of_OI_Other_Rept_Spread_Other', 'Pct_of_OI_Tot_Rept_Long_Other', 'Pct_of_OI_Tot_Rept_Short_Other', 'Pct_of_OI_NonRept_Long_Other', 'Pct_of_OI_NonRept_Short_Other', 'Traders_Tot_All', 'Traders_Prod_Merc_Long_All', 'Traders_Prod_Merc_Short_All', 'Traders_Swap_Long_All', 'Traders_Swap_Short_All', 'Traders_Swap_Spread_All', 'Traders_M_Money_Long_All', 'Traders_M_Money_Short_All', 'Traders_M_Money_Spread_All', 'Traders_Other_Rept_Long_All', 'Traders_Other_Rept_Short_All', 'Traders_Other_Rept_Spread_All', 'Traders_Tot_Rept_Long_All', 'Traders_Tot_Rept_Short_All', 'Traders_Tot_
]
}
],
"source": [
"print(list(silver_df.columns))"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "b30a8b94-6388-4476-a008-8e69959cfbb9",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['Pct_of_Open_Interest_Old',\n",
" 'Pct_of_OI_Prod_Merc_Long_Old',\n",
" 'Pct_of_OI_Prod_Merc_Short_Old',\n",
" 'Pct_of_OI_Swap_Long_Old',\n",
" 'Pct_of_OI_Swap_Short_Old',\n",
" 'Pct_of_OI_Swap_Spread_Old',\n",
" 'Pct_of_OI_M_Money_Long_Old',\n",
" 'Pct_of_OI_M_Money_Short_Old',\n",
" 'Pct_of_OI_M_Money_Spread_Old',\n",
" 'Pct_of_OI_Other_Rept_Long_Old',\n",
" 'Pct_of_OI_Other_Rept_Short_Old',\n",
" 'Pct_of_OI_Other_Rept_Spread_Old',\n",
" 'Pct_of_OI_Tot_Rept_Long_Old',\n",
" 'Pct_of_OI_Tot_Rept_Short_Old',\n",
" 'Pct_of_OI_NonRept_Long_Old',\n",
" 'Pct_of_OI_NonRept_Short_Old']"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"open_interest_pct_old_columns = [col for col in df.columns if 'pct' in col.lower() and col.endswith('_Old')]\n",
"open_interest_pct_old_columns"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "7f04ca58-7f5b-4cef-9941-620067cd8751",
"metadata": {},
"outputs": [],
"source": [
"needed_columns = [\"Open_Interest_All\", \"Pct_of_OI_Prod_Merc_Long_Old\", \"Pct_of_OI_Prod_Merc_Short_Old\"]"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "a862cbe7-49c8-43b1-ba34-a87f37b40daa",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Open_Interest_All</th>\n",
" <th>Pct_of_OI_Prod_Merc_Long_Old</th>\n",
" <th>Pct_of_OI_Prod_Merc_Short_Old</th>\n",
" </tr>\n",
" <tr>\n",
" <th>Report_Date_as_YYYY-MM-DD</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2024-08-13</th>\n",
" <td>147859</td>\n",
" <td>3.1</td>\n",
" <td>25.3</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2024-08-06</th>\n",
" <td>147537</td>\n",
" <td>3.0</td>\n",
" <td>25.4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2024-07-30</th>\n",
" <td>151437</td>\n",
" <td>2.7</td>\n",
" <td>25.4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2024-07-23</th>\n",
" <td>157106</td>\n",
" <td>2.2</td>\n",
" <td>25.8</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2024-07-16</th>\n",
" <td>166641</td>\n",
" <td>1.7</td>\n",
" <td>26.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Open_Interest_All Pct_of_OI_Prod_Merc_Long_Old \\\n",
"Report_Date_as_YYYY-MM-DD \n",
"2024-08-13 147859 3.1 \n",
"2024-08-06 147537 3.0 \n",
"2024-07-30 151437 2.7 \n",
"2024-07-23 157106 2.2 \n",
"2024-07-16 166641 1.7 \n",
"\n",
" Pct_of_OI_Prod_Merc_Short_Old \n",
"Report_Date_as_YYYY-MM-DD \n",
"2024-08-13 25.3 \n",
"2024-08-06 25.4 \n",
"2024-07-30 25.4 \n",
"2024-07-23 25.8 \n",
"2024-07-16 26.0 "
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"filtered_silver_df = silver_df[needed_columns].copy()\n",
"filtered_silver_df.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "106e2cd2-8c1a-407e-9c1d-94301b3261e7",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "ac29c5fd-e220-4c6c-9366-ce3349a71385",
"metadata": {},
"source": [
"## Open Interest (COT) CFTC"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "6c517b33-fae5-44f2-be07-00b7e9a65cac",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"plotlyServerURL": "https://plot.ly"
},
"data": [
{
"line": {
"color": "blue",
"width": 2
},
"mode": "lines",
"name": "Open Interest",
"type": "scatter",
"x": [
"2024-08-13T00:00:00",
"2024-08-06T00:00:00",
"2024-07-30T00:00:00",
"2024-07-23T00:00:00",
"2024-07-16T00:00:00",
"2024-07-09T00:00:00",
"2024-07-02T00:00:00",
"2024-06-25T00:00:00",
"2024-06-18T00:00:00",
"2024-06-11T00:00:00",
"2024-06-04T00:00:00",
"2024-05-28T00:00:00",
"2024-05-21T00:00:00",
"2024-05-14T00:00:00",
"2024-05-07T00:00:00",
"2024-04-30T00:00:00",
"2024-04-23T00:00:00",
"2024-04-16T00:00:00",
"2024-04-09T00:00:00",
"2024-04-02T00:00:00",
"2024-03-26T00:00:00",
"2024-03-19T00:00:00",
"2024-03-12T00:00:00",
"2024-03-05T00:00:00",
"2024-02-27T00:00:00",
"2024-02-20T00:00:00",
"2024-02-13T00:00:00",
"2024-02-06T00:00:00",
"2024-01-30T00:00:00",
"2024-01-23T00:00:00",
"2024-01-16T00:00:00",
"2024-01-09T00:00:00",
"2024-01-02T00:00:00"
],
"y": [
147859,
147537,
151437,
157106,
166641,
163245,
154724,
166470,
176605,
176036,
179854,
184846,
186945,
171371,
161868,
166315,
175740,
175784,
174397,
166823,
160327,
153154,
144534,
141026,
144382,
146584,
152544,
147333,
136544,
138617,
131956,
131992,
134725
]
}
],
"layout": {
"height": 600,
"hovermode": "x unified",
"template": {
"data": {
"bar": [
{
"error_x": {
"color": "#2a3f5f"
},
"error_y": {
"color": "#2a3f5f"
},
"marker": {
"line": {
"color": "#E5ECF6",
"width": 0.5
},
"pattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
}
},
"type": "bar"
}
],
"barpolar": [
{
"marker": {
"line": {
"color": "#E5ECF6",
"width": 0.5
},
"pattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
}
},
"type": "barpolar"
}
],
"carpet": [
{
"aaxis": {
"endlinecolor": "#2a3f5f",
"gridcolor": "white",
"linecolor": "white",
"minorgridcolor": "white",
"startlinecolor": "#2a3f5f"
},
"baxis": {
"endlinecolor": "#2a3f5f",
"gridcolor": "white",
"linecolor": "white",
"minorgridcolor": "white",
"startlinecolor": "#2a3f5f"
},
"type": "carpet"
}
],
"choropleth": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "choropleth"
}
],
"contour": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "contour"
}
],
"contourcarpet": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "contourcarpet"
}
],
"heatmap": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "heatmap"
}
],
"heatmapgl": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "heatmapgl"
}
],
"histogram": [
{
"marker": {
"pattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
}
},
"type": "histogram"
}
],
"histogram2d": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "histogram2d"
}
],
"histogram2dcontour": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "histogram2dcontour"
}
],
"mesh3d": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "mesh3d"
}
],
"parcoords": [
{
"line": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "parcoords"
}
],
"pie": [
{
"automargin": true,
"type": "pie"
}
],
"scatter": [
{
"fillpattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
},
"type": "scatter"
}
],
"scatter3d": [
{
"line": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatter3d"
}
],
"scattercarpet": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattercarpet"
}
],
"scattergeo": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattergeo"
}
],
"scattergl": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattergl"
}
],
"scattermapbox": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattermapbox"
}
],
"scatterpolar": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterpolar"
}
],
"scatterpolargl": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterpolargl"
}
],
"scatterternary": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterternary"
}
],
"surface": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "surface"
}
],
"table": [
{
"cells": {
"fill": {
"color": "#EBF0F8"
},
"line": {
"color": "white"
}
},
"header": {
"fill": {
"color": "#C8D4E3"
},
"line": {
"color": "white"
}
},
"type": "table"
}
]
},
"layout": {
"annotationdefaults": {
"arrowcolor": "#2a3f5f",
"arrowhead": 0,
"arrowwidth": 1
},
"autotypenumbers": "strict",
"coloraxis": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"colorscale": {
"diverging": [
[
0,
"#8e0152"
],
[
0.1,
"#c51b7d"
],
[
0.2,
"#de77ae"
],
[
0.3,
"#f1b6da"
],
[
0.4,
"#fde0ef"
],
[
0.5,
"#f7f7f7"
],
[
0.6,
"#e6f5d0"
],
[
0.7,
"#b8e186"
],
[
0.8,
"#7fbc41"
],
[
0.9,
"#4d9221"
],
[
1,
"#276419"
]
],
"sequential": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"sequentialminus": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
]
},
"colorway": [
"#636efa",
"#EF553B",
"#00cc96",
"#ab63fa",
"#FFA15A",
"#19d3f3",
"#FF6692",
"#B6E880",
"#FF97FF",
"#FECB52"
],
"font": {
"color": "#2a3f5f"
},
"geo": {
"bgcolor": "white",
"lakecolor": "white",
"landcolor": "#E5ECF6",
"showlakes": true,
"showland": true,
"subunitcolor": "white"
},
"hoverlabel": {
"align": "left"
},
"hovermode": "closest",
"mapbox": {
"style": "light"
},
"paper_bgcolor": "white",
"plot_bgcolor": "#E5ECF6",
"polar": {
"angularaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"bgcolor": "#E5ECF6",
"radialaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
}
},
"scene": {
"xaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
},
"yaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
},
"zaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
}
},
"shapedefaults": {
"line": {
"color": "#2a3f5f"
}
},
"ternary": {
"aaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"baxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"bgcolor": "#E5ECF6",
"caxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
}
},
"title": {
"x": 0.05
},
"xaxis": {
"automargin": true,
"gridcolor": "white",
"linecolor": "white",
"ticks": "",
"title": {
"standoff": 15
},
"zerolinecolor": "white",
"zerolinewidth": 2
},
"yaxis": {
"automargin": true,
"gridcolor": "white",
"linecolor": "white",
"ticks": "",
"title": {
"standoff": 15
},
"zerolinecolor": "white",
"zerolinewidth": 2
}
}
},
"title": {
"text": "COT Data: Open Interest Over Time (Silver)"
},
"width": 800,
"xaxis": {
"autorange": true,
"dtick": "M1",
"range": [
"2024-01-02",
"2024-08-13"
],
"tickangle": 45,
"tickformat": "%d %b %Y",
"ticklabelmode": "period",
"title": {
"text": "Date"
},
"type": "date"
},
"yaxis": {
"autorange": true,
"range": [
128901.05555555556,
189999.94444444444
],
"title": {
"text": "Open Interest"
},
"type": "linear"
}
}
},
"image/png": "iVBORw0KGgoAAAANSUhEUgAABJUAAAJYCAYAAADbkteOAAAAAXNSR0IArs4c6QAAIABJREFUeF7snQm4VdP7x79nuPeekKRJIjJTJJpEI5IUSqEMjZTSnIpEM0WDUqmkiZKSoSikUZQpZJ6FJj+Jpjudc/7/d1/7ure695599hn2Xuu7n8eT7llr7bU+73vuPufTGjzhcDgMXiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRggYCHUskCLRYlARIgARIgARIgARIgARIgARIgARIgARIwCFAqMRFIgARIgARIgARIgARIgARIgARIgARIgAQsE6BUsoyMFUiABEiABEiABEiABEiABEiABEiABEiABCiVmAMkQAIkQAIkQAIkQAIkQAIkQAIkQAIkQAKWCVAqWUbGCiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAApRKzAESIAESIAESIAESIAESIAESIAESIAESIAHLBCiVLCNjBRIgARIgARIgARIgARIgARIgARIgARIgAUol5gAJkAAJkAAJkAAJkAAJkAAJkAAJkAAJkIBlApRKlpGxAgmQAAmQAAmQAAmQAAmQAAmQAAmQAAmQAKUSc4AESIAESIAESIAESIAESIAESIAESIAESMAyAUoly8hYgQRIgARIgARIgARIgARIgARIgARIgARIgFKJOUACJEACJEACJEACJEACJEACJEACJEACJGCZAKWSZWSsQAIkQAIkQAIkQAIkQAIkQAIkQAIkQAIkQKnEHCABEiABEiABEiABEiABEiABEiABEiABErBMgFLJMjJWIAESIAESIAESIAESIAESIAESIAESIAESoFRiDpAACZAACZAACZAACZAACZAACZAACZAACVgmQKlkGRkrkAAJkAAJkAAJkAAJkAAJkAAJkAAJkAAJUCoxB0iABEiABEiABEiABEiABEiABEiABEiABCwToFSyjIwVSIAESIAESIAESIAESIAESIAESIAESIAEKJWYAyRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAApYJUCpZRsYKJEACJEACJEACJEACJEACJEACJEACJEAClErMARIgARIgARIgARIgARIgARIgARIgARIgAcsEKJUsI2MFEiABEiABEiABEiABEiABEiABEiABEiABSiXmAAmQAAmQAAmQAAmQAAmQAAmQAAmQAAmQgGUClEqWkbECCZAACZAACZAACZAACZAACZAACZAACZAApRJzgARIgARIgARIgARIgARIgARIgARIgARIwDIBSiXLyFiBBEiABEiABEiABEiABEiABEiABEiABEiAUok5QAIkQAIkQAIkQAIkQAIkQAIkQAIkQAIkYJkApZJlZKxAAiRAAiRAAiRAAiRAAiRAAiRAAiRAAiRAqcQcIAESIAESIAESIAESIAESIAESIAESIAESsEyAUskyMlYgARIgARIgARIgARIgARIgARIgARIgARKgVGIOkAAJkAAJkAAJkAAJkAAJkAAJkAAJkAAJWCZAqWQZGSuQAAmQAAmQAAmQAAmQAAmQAAmQAAmQAAlQKjEHSIAESIAESIAESIAESIAESIAESIAESIAELBOgVLKMjBVIgARIgARIgARIgARIgARIgARIgARIgAS0l0pZ2UHs2PU/pKWmotSJx8Pv8xWYFeFwGLv/txf7Dx7CqeXLIDU1hRlkg8Df+w5g1x9/4eRypXDcscVstMSqbiCQHQxi+87/wev14uRypeH1etzQ7UL7KL8TDhxMh8/nQ7FAqiPHs+iV1ThwKB3XNKiJCieVztdH6f+evfvwz74DOKHEcShR/Lgj4pKRmYWsrGwEAqm5vx+Xv/Ue/vp7H+5o1TjhY5b+LFi6CmGEccv1jXDsMYGE94E3JAESIAESIAESIAESIAESyCGgrVR6c92HePKZpfjhl+35cqFBnYvR4tq6aHT5JblfrkR+PDb1ebyx9gMcPJSeW/78s0/DAz1vxyUXnm38bOyUhZi7+I2IcmvT8qkoftwxRy27ePlaDH18Tr7XTjyhOM48vQKuu6o2mjSoWWDdom6+7fddWLxsHepfVhXVq55bVPGYvy5iYe4Lb2DOohXGl1nzKlemJLq3b4GWTevC43G2bPjmh1/RstOQfGxOO6UcXn92jCVewWAIE2cuwRmnlTdyzumXyIkdu/eg912tLHX1x207MHLCPGze8tUR77UHe9+J8mVPtNRePAuLKLnihh4R3aJz2+vQunkDXNPmPlQ5txIWTX84onqJLLRqw0foNWQyWjath2H9O+T+TjuUnok5L6zAMwtX5PudJn27vEYVY1xX16tudHXI2Gew9PX1eGpMP9StdaHxs9vvHYUtn3+HL9bm/z2VqLGZvyM7tWmKvl1uTtRteR8SIAESIAESIAESIAESIIHDCGgplQaNnoFlb75roLit5dXGl/o//tyLT7/8Ae99+IXx8w9WPIVjigWMmRW3dB1mCBCRSFfWvcT41/zPvvoht42H+7bDzdc3hPzr/Xsf5dSXa+8/+7H23U8gwuSySyvnQy9fpgua2fDCq2swbPxc1Lj4PJxdqQL+2X/QmNHzwSdfG22cUr4MFk57CCKarF7vb/kaHfo8igHd26Bd62usVrdVXoRS536PGeOQvt/YpC5OrVAWP/6yHS++tt74cnvdlbUxdkhXW/eJd2URDytWv48Vqzfj463fGrM1Kp9zOpo3rmPp1jL74+KrO0NE5pTRvS3VTUbhO3qMNsZrRSRs2PwZug4cb3S30eXVjJyWPHhn81ZDMsl77JnxA3Dh+WckY0hH3FNmHY2e9Gy+n8t7Wt5/0v/jix+b+9oVNS80xiPS5fRTT8LA7m0cMQazEwcPZaDZnYNQovixWDxzWL5ZmBNmLMbTC14z3ocN6lTDOWecAhHOWz7/Hl9994shj0QiyTV/yZt498MvcG+HFqh87unGz5ItlaQPDz8+G0uWr8Orc0YZwp0XCZAACZAACZAACZAACZBA4gloJ5XML7kiemaNG4BKFcvno77uvU/Rf/g0rFs60fjCa/4rvQiD4f075FvytnrjFvQY/IRRbtWicShx/H9fOKVRmQV1fbsHcGOTKzBqUOeIo2tKpdH334Ubrrk8t558sR0zZSHeWPs+LrrgTDwzfqDlJTfJlEoi8kToyRfY6WP7o2zpE3LH9tO2HejUb6zx5X3W+AGofckFEfOKRUFZBmR1hpTEYt7iN7Bg6hBUveBMy92wIpWi6Z/lDhVRwapUEnl03e2D8NuOP/Bg7zvQ5sYrc+8QCoUxadaLmPncclxy4TmYP/mBWHc3Zu0NGPEUXnt7E1Y8NwYVK5SLWbvxbkj4Tp+/zPg9V/vS/95P3//0O27oMNh4Hy6Y+tARv0NkRuamj7+EyPKCrlhJJTt5Lf8Q0OCm3qhV7Xzjd4bV92+8+bN9EiABEiABEiABEiABEtCBgHZSqentA/HLb7vw1Ji+qFvroqPGWGYGFT+2mPFluEnbAYY0WvvixKPu3THqiflY8NLb6N6hBbq1uyFfe7GWStK4iIjuD0zExg8+z3fPz778AdPmvWrM+pF+S58vPK8S7mjdGA3rVDP6JWVGPfEsPv/mJ2O20xmnnWz8XJbv3XVbM+P1otrIO0ARatnBEAb3ut1or7BLvjxeeXNfQxoVJGHky2zfoVPySYYvvvkZT85+yRBzP/+6EyvXbMa3P/6GalXORr+uNxt/5r2E+RNPL8GWrd8Zs8vk9Xva3WAs6TGvl1ZsgCx/7NGxBZa99R5Wv/Oxweyy6pUxuOftR4jGgsZVkFSSn8s4u3e4EVNmvwwRmXI1aVgTA7rdaixdlFlZvR960oijxMpcilgskIbxQ7sZ5UXKzF/8JlaueT83ZvVqX4SenW7KXf6YnpGJPg8Ls7ON2XJLX1uPrV//aLwuy53kioTJ//b8jVkLX8fG97ca5WU5n7C79YZGxiwiyfOXV240+l2vdtVcJCKLDt+nx3xRZu4NHDXdYD/jsf5HYBSxdGOHwcb9npkw0JADshxQlhcO6X0HTj5s/x9ZgvXW+o/Qrf2NRm7LJTMBRext/fon4++
"text/html": [
"<div> <div id=\"4682477c-df05-4b78-8607-c7bf85a839c3\" class=\"plotly-graph-div\" style=\"height:600px; width:800px;\"></div> <script type=\"text/javascript\"> require([\"plotly\"], function(Plotly) { window.PLOTLYENV=window.PLOTLYENV || {}; if (document.getElementById(\"4682477c-df05-4b78-8607-c7bf85a839c3\")) { Plotly.newPlot( \"4682477c-df05-4b78-8607-c7bf85a839c3\", [{\"line\":{\"color\":\"blue\",\"width\":2},\"mode\":\"lines\",\"name\":\"Open Interest\",\"x\":[\"2024-08-13T00:00:00\",\"2024-08-06T00:00:00\",\"2024-07-30T00:00:00\",\"2024-07-23T00:00:00\",\"2024-07-16T00:00:00\",\"2024-07-09T00:00:00\",\"2024-07-02T00:00:00\",\"2024-06-25T00:00:00\",\"2024-06-18T00:00:00\",\"2024-06-11T00:00:00\",\"2024-06-04T00:00:00\",\"2024-05-28T00:00:00\",\"2024-05-21T00:00:00\",\"2024-05-14T00:00:00\",\"2024-05-07T00:00:00\",\"2024-04-30T00:00:00\",\"2024-04-23T00:00:00\",\"2024-04-16T00:00:00\",\"2024-04-09T00:00:00\",\"2024-04-02T00:00:00\",\"2024-03-26T00:00:00\",\"2024-03-19T00:00:00\",\"2024-03-12T00:00:00\",\"2024-03-05T00:00:00\",\"2024-02-27T00:00:00\",\"2024-02-20T00:00:00\",\"2024-02-13T00:00:00\",\"2024-02-06T00:00:00\",\"2024-01-30T00:00:00\",\"2024-01-23T00:00:00\",\"2024-01-16T00:00:00\",\"2024-01-09T00:00:00\",\"2024-01-02T00:00:00\"],\"y\":[147859,147537,151437,157106,166641,163245,154724,166470,176605,176036,179854,184846,186945,171371,161868,166315,175740,175784,174397,166823,160327,153154,144534,141026,144382,146584,152544,147333,136544,138617,131956,131992,134725],\"type\":\"scatter\"}], {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0
" \n",
"var gd = document.getElementById('4682477c-df05-4b78-8607-c7bf85a839c3');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" }) }; }); </script> </div>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Metrics for Open Interest:\n",
"Volatility (Standard Deviation): 15889.35\n",
"Average: 158040.03\n",
"Median: 157106.00\n",
"Minimum: 131956.00\n",
"Maximum: 186945.00\n",
"Typical Range: 146584.00 to 171371.00\n",
"\n",
"Overall trend: Downward\n",
"\n",
"Significant points:\n",
"Peak on 4: 166641.00\n",
"Peak on 12: 186945.00\n",
"Peak on 17: 175784.00\n",
"Peak on 26: 152544.00\n",
"Peak on 29: 138617.00\n",
"Trough on 6: 154724.00\n",
"Trough on 14: 161868.00\n",
"Trough on 23: 141026.00\n",
"Trough on 28: 136544.00\n",
"Trough on 30: 131956.00\n",
"\n",
"Sudden changes (>5% day-to-day):\n",
"2024-07-16 00:00:00: 6.07% change\n",
"2024-07-02 00:00:00: -5.22% change\n",
"2024-06-25 00:00:00: 7.59% change\n",
"2024-06-18 00:00:00: 6.09% change\n",
"2024-05-14 00:00:00: -8.33% change\n",
"2024-05-07 00:00:00: -5.55% change\n",
"2024-04-23 00:00:00: 5.67% change\n",
"2024-03-12 00:00:00: -5.63% change\n",
"2024-01-30 00:00:00: -7.32% change\n"
]
}
],
"source": [
"import pandas as pd\n",
"import plotly.graph_objects as go\n",
"import numpy as np\n",
"from scipy import signal\n",
"\n",
"# Assuming df is your DataFrame with the COT data\n",
"# If not, you'll need to load your data here\n",
"\n",
"# Create the main figure\n",
"fig = go.Figure()\n",
"\n",
"# Add trace for Open Interest\n",
"fig.add_trace(go.Scatter(\n",
" x=filtered_silver_df.index, \n",
" y=filtered_silver_df['Open_Interest_All'], \n",
" mode='lines',\n",
" name='Open Interest',\n",
" line=dict(width=2, color='blue')\n",
"))\n",
"\n",
"# Update layout\n",
"fig.update_layout(\n",
" title='COT Data: Open Interest Over Time (Silver)',\n",
" yaxis_title='Open Interest',\n",
" xaxis_title='Date',\n",
" height=600,\n",
" width=800,\n",
" hovermode='x unified'\n",
")\n",
"\n",
"# Improve date labeling for better readability\n",
"fig.update_xaxes(\n",
" tickformat=\"%d %b %Y\",\n",
" tickangle=45,\n",
" dtick=\"M1\",\n",
" ticklabelmode=\"period\"\n",
")\n",
"\n",
"# Show the plot\n",
"fig.show()\n",
"\n",
"# Calculate and print metrics\n",
"print(\"\\nMetrics for Open Interest:\")\n",
"data = filtered_silver_df['Open_Interest_All']\n",
"\n",
"print(f\"Volatility (Standard Deviation): {data.std():.2f}\")\n",
"print(f\"Average: {data.mean():.2f}\")\n",
"print(f\"Median: {data.median():.2f}\")\n",
"print(f\"Minimum: {data.min():.2f}\")\n",
"print(f\"Maximum: {data.max():.2f}\")\n",
"\n",
"# Calculate typical range\n",
"typical_low = np.percentile(data, 25)\n",
"typical_high = np.percentile(data, 75)\n",
"print(f\"Typical Range: {typical_low:.2f} to {typical_high:.2f}\")\n",
"\n",
"# Identify trends\n",
"def identify_trend(series):\n",
" # Calculate the overall trend\n",
" trend = np.polyfit(range(len(series)), series, 1)[0]\n",
" if trend > 0:\n",
" return \"Upward\"\n",
" elif trend < 0:\n",
" return \"Downward\"\n",
" else:\n",
" return \"Stable\"\n",
"\n",
"trend = identify_trend(data)\n",
"print(f\"\\nOverall trend: {trend}\")\n",
"\n",
"# Identify significant points\n",
"def find_peaks(series, prominence=1000):\n",
" peaks, _ = signal.find_peaks(series, prominence=prominence)\n",
" troughs, _ = signal.find_peaks(-series, prominence=prominence)\n",
" return peaks, troughs\n",
"\n",
"peaks, troughs = find_peaks(data)\n",
"\n",
"print(\"\\nSignificant points:\")\n",
"for peak in peaks:\n",
" print(f\"Peak on {df.index[peak]}: {data.iloc[peak]:.2f}\")\n",
"for trough in troughs:\n",
" print(f\"Trough on {df.index[trough]}: {data.iloc[trough]:.2f}\")\n",
"\n",
"# Identify sudden changes\n",
"pct_change = data.pct_change()\n",
"sudden_changes = pct_change[abs(pct_change) > 0.05] # 5% threshold\n",
"if not sudden_changes.empty:\n",
" print(\"\\nSudden changes (>5% day-to-day):\")\n",
" for date, change in sudden_changes.items():\n",
" print(f\"{date}: {change*100:.2f}% change\")\n",
"else:\n",
" print(\"\\nNo sudden changes above 5% threshold detected.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ce80decc-23b4-4ae5-b112-1e433440a05f",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "ede29b5a-eb59-448c-a7d6-bba51d6dee84",
"metadata": {},
"source": [
"## Data Export for AmiBroker\n",
"\n",
"(You have to tick the box \"No quotation data\" in the Import wizard and select the fields according to the CSV header)"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "d7d3aae5-7043-409e-b5ae-0aed4a3b4de4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Data exported to silver_data_for_amibroker.csv\n"
]
}
],
"source": [
"import pandas as pd\n",
"\n",
"def export_to_amibroker_csv(df, output_file='amibroker_data.csv'):\n",
" # Create a copy of the DataFrame to avoid modifying the original\n",
" export_df = df.copy()\n",
" \n",
" # Reset the index to make the date a column\n",
" export_df = export_df.reset_index()\n",
" \n",
" # Rename columns to match AmiBroker format\n",
" export_df['Date'] = pd.to_datetime(export_df['Report_Date_as_YYYY-MM-DD'])\n",
" export_df['Close'] = export_df['Open_Interest_All']\n",
" export_df['Volume'] = export_df['Pct_of_OI_Prod_Merc_Long_Old']\n",
" \n",
" # Create Open, High, Low columns with the same value as Close\n",
" export_df['Open'] = export_df['Close']\n",
" export_df['High'] = export_df['Close']\n",
" export_df['Low'] = export_df['Close']\n",
" \n",
" # Select and order columns for AmiBroker\n",
" amibroker_df = export_df[['Date', 'Open', 'High', 'Low', 'Close', 'Volume']]\n",
" \n",
" # Sort by date in descending order (most recent first)\n",
" amibroker_df = amibroker_df.sort_values('Date', ascending=False)\n",
" \n",
" # Export to CSV\n",
" amibroker_df.to_csv(output_file, index=False, date_format='%Y-%m-%d')\n",
" print(f\"Data exported to {output_file}\")\n",
"\n",
"# Usage\n",
"export_to_amibroker_csv(filtered_silver_df, 'silver_data_for_amibroker.csv')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2044602a-9024-4522-bc15-ef8903c28758",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in New Issue Copy Permalink