SimpleFastDB
The SimpleFastDB is an in-memory database which contains all time series data used by Collections and Portfolios. This data usually consist of raw external data or trivially modified such. This external data is commonly obtained through the ExternalDataHandler.
Default SimpleFastDB
A default SimpleFastDB is automatically created and used by TFire. This default database is initialized when TFire is started, ensuring that there's always a database available for storing and retrieving time series data.
All SimpleFastDB databases are stored in TFire.DBStore
and can be accessed through TFire.DBStore.dbs
. The first (and usually only) database in this vector is the default SimpleFastDB. If no SimpleFastDB is explicitly specified for functions that take a SimpleFastDB as an argument, the default SimpleFastDB is used.
For most use cases, working with the default SimpleFastDB is sufficient and convenient, as it's automatically available and integrated with other TFire components. However, if you need to create additional databases or want to work with a specific database instance, you can do so.
TimeSeries
The SimpleFastDB stores TimeSeries objects. A TimeSeries object contains a number of continuous time series for one ticker
. Thus if times series data from several discontinuous times are stored in the SimpleFastDB, they will be stored as separate TimeSeries objects. Currently only daily discrete time series are supported.
Every TimeSeries contains an index
vector of the associated datetimes. Further, it supports any ammount of DataFields where every such DataField represents a vector of values. To access a DataField the one may use the function
get_datafield(ts::TimeSeries, field::Symbol)
A list of the available DataFields can be obtained with
Creating a SimpleFastDB
A new SimpleFastDB is created with the function
Besides the obligatory Specification argument this function takes the following optional arguments:
read_external
- Bool, if true the SimpleFastDB will read external data from online sources.read_disk
- Bool, if true the SimpleFastDB will read data from disk.data_path
- String, the path to the data file(s) to read ifread_disk=true
.internet_service
- String, the internet service to use ifread_external=true
.
Accessing a TimeSeries
Time series will usually be accessed through Collections but may also be accessed directly from a SimpleFastDB. Access to a TimeSeries is carried out with the function
get_time_series(sfdb::SimpleFastDB, resolution::String, interval::String, ticker::String; date=Date(0001,1,1))
method is used.
resolution
- String, the resolution of the TimeSeries.interval
- String, "continuous" or "1d". Specifies if the TimeSeries is continuous or daily (like a daily traded stock on some resolution finer than daily).ticker
- String, the ticker of the TimeSeries.date
- Date, the date of the TimeSeries in caseinterval="1d"
.
Adding Data to a SimpleFastDB
There are two methods to add data to a SimpleFastDB: from disk storage or from internet sources. Both require a Specification that defines what data should be added.
I'll revise the section to highlight how the External Data Handler (EDH) is used in the data addition process. Here's the updated version:
Adding Data to a SimpleFastDB
There are two methods to add data to a SimpleFastDB: from disk storage or from internet sources. Both methods work through the External Data Handler (EDH) which manages data fetching, processing, and integration.
Adding Data From Disk
Use add_data_from_disk!
to load data from local files via the External Data Handler:
# Create a specification
spec = Specification(["AAPL", "MSFT"], Date(2020,1,1), Date(2023,12,31))
# Basic usage with default SimpleFastDB
add_data_from_disk!(spec, "path/to/data.jld2")
The method workflow:
- Converts the specification to an external data specification
- Uses the External Data Handler to fetch and process the data from disk
- The EDH handles file reading and data parsing
- Processed data is added to the specified SimpleFastDB (or default if none specified)
Adding Data From the Internet
Use add_data_from_internet!
to fetch and add data from online sources through the External Data Handler:
# Basic usage with default settings
add_data_from_internet!(spec)
# With a specific internet service
add_data_from_internet!(spec, internet_service=YahooFinance)
The method workflow:
- Converts the specification to an external data specification
- The External Data Handler manages the connection to the specified internet service
- The EDH handles data downloading, validation, and preprocessing
- Processed data is added to the specified SimpleFastDB (or default if none specified)
Both methods use the External Data Handler to ensure:
- Proper data format conversion
- Consistent data cleaning and preparation
- Error handling and validation
- Correct integration with the SimpleFastDB
The default SimpleFastDB (first database in DBStore.dbs
) will be used unless another SimpleFastDB instance is explicitly specified through the sfdb
parameter. Both methods also support an optional trading_dates
parameter to filter the data based on specific trading dates.
For more details about the External Data Handler, see ExternalDataHandler.
Functions
For functions related to the Collection see SimpleFastDB - Functions.