ExternalDataHandler
The External Data Handler is the submodule that handles fetching data from verious internet sources as well as the loading and saving of data from local disk storage. In most cases, users don't need to manually interact with the External Data Handler (EDH). The EDH is then automatically utilized to ensure that all required data is available in the SimpleFastDB when creating and managing collections.
Basic Usage
The three primary functions for interacting with external data are:
Technicalities
ExternalData
A mutable structure that encapsulates the raw data for a given ticker. It contains:
data: A Dict holding the actual data.ticker: Symbol or name identifying the data.resolution: Describes the granularity of the data.fromandto: Defines the time interval for the data. The actual time series might not include data for the whole period if some is missing/nonexistant.
ExternalDataCollection
This structure holds a collection of ExternalData structures.
External Data File
The primary tool/format used to save and retrieve external data is the ExternalDataFile. This struct is designed to consolidate different external data types and provide a consistent way of accessing them.
The ExternalDataFile is composed of:
- A specification, which describes the properties of the data (e.g., resolution and file format version).
- Tickers, which associate specific symbols with their corresponding ExternalData.
Resolution and Directory Structure
Data resolution determines where data is stored and accessed. The resolution_to_dir function helps translate resolution to the appropriate directory structure, ensuring consistent organization and easy retrieval. The currentrly supported resolutions are daily (1d) and minute (1m).