SimpleFastDB

The SimpleFastDB is an in-memory database which contains all time series data used by Collections and Portfolios. This data usually consist of raw external data or trivially modified such. This external data is commonly obtained through the ExternalDataHandler.

Default SimpleFastDB

A default SimpleFastDB is automatically created and used by TFire. This default database is initialized when TFire is started, ensuring that there's always a database available for storing and retrieving time series data.

All SimpleFastDB databases are stored in TFire.DBStore and can be accessed through TFire.DBStore.dbs. The first (and usually only) database in this vector is the default SimpleFastDB. If no SimpleFastDB is explicitly specified for functions that take a SimpleFastDB as an argument, the default SimpleFastDB is used.

For most use cases, working with the default SimpleFastDB is sufficient and convenient, as it's automatically available and integrated with other TFire components. However, if you need to create additional databases or want to work with a specific database instance, you can do so.

TimeSeries

The SimpleFastDB stores TimeSeries objects. A TimeSeries object contains a number of continuous time series for one ticker. Thus if times series data from several discontinuous times are stored in the SimpleFastDB, they will be stored as separate TimeSeries objects. Currently only daily discrete time series are supported.

Every TimeSeries contains an index vector of the associated datetimes. Further, it supports any ammount of DataFields where every such DataField represents a vector of values. To access a DataField the one may use the function

get_datafield(ts::TimeSeries, field::Symbol)

A list of the available DataFields can be obtained with

DataFields(ts::TimeSeries)

Creating a SimpleFastDB

A new SimpleFastDB is created with the function

setup_simple_fast_db(spec::Specification; read_external=true, read_disk=false, data_path = "", internet_service=YahooData)

Besides the obligatory Specification argument this function takes the following optional arguments:

  • read_external - Bool, if true the SimpleFastDB will read external data from online sources.
  • read_disk - Bool, if true the SimpleFastDB will read data from disk.
  • data_path - String, the path to the data file(s) to read if read_disk=true.
  • internet_service - String, the internet service to use if read_external=true.

Accessing a TimeSeries

Time series will usually be accessed through Collections but may also be accessed directly from a SimpleFastDB. Access to a TimeSeries is carried out with the function

get_time_series(sfdb::SimpleFastDB, resolution::String, interval::String, ticker::String; date=Date(0001,1,1)) method is used.

  • resolution - String, the resolution of the TimeSeries.
  • interval - String, "continuous" or "1d". Specifies if the TimeSeries is continuous or daily (like a daily traded stock on some resolution finer than daily).
  • ticker - String, the ticker of the TimeSeries.
  • date - Date, the date of the TimeSeries in case interval="1d".

Adding Data to a SimpleFastDB

There are two methods to add data to a SimpleFastDB: from disk storage or from internet sources. Both require a Specification that defines what data should be added.

I'll revise the section to highlight how the External Data Handler (EDH) is used in the data addition process. Here's the updated version:

Adding Data to a SimpleFastDB

There are two methods to add data to a SimpleFastDB: from disk storage or from internet sources. Both methods work through the External Data Handler (EDH) which manages data fetching, processing, and integration.

Adding Data From Disk

Use add_data_from_disk! to load data from local files via the External Data Handler:

# Create a specification
spec = Specification(["AAPL", "MSFT"], Date(2020,1,1), Date(2023,12,31))

# Basic usage with default SimpleFastDB
add_data_from_disk!(spec, "path/to/data.jld2")

The method workflow:

  1. Converts the specification to an external data specification
  2. Uses the External Data Handler to fetch and process the data from disk
  3. The EDH handles file reading and data parsing
  4. Processed data is added to the specified SimpleFastDB (or default if none specified)

Adding Data From the Internet

Use add_data_from_internet! to fetch and add data from online sources through the External Data Handler:

# Basic usage with default settings
add_data_from_internet!(spec)

# With a specific internet service
add_data_from_internet!(spec, internet_service=YahooFinance)

The method workflow:

  1. Converts the specification to an external data specification
  2. The External Data Handler manages the connection to the specified internet service
  3. The EDH handles data downloading, validation, and preprocessing
  4. Processed data is added to the specified SimpleFastDB (or default if none specified)

Both methods use the External Data Handler to ensure:

  • Proper data format conversion
  • Consistent data cleaning and preparation
  • Error handling and validation
  • Correct integration with the SimpleFastDB

The default SimpleFastDB (first database in DBStore.dbs) will be used unless another SimpleFastDB instance is explicitly specified through the sfdb parameter. Both methods also support an optional trading_dates parameter to filter the data based on specific trading dates.

For more details about the External Data Handler, see ExternalDataHandler.

Functions

For functions related to the Collection see SimpleFastDB - Functions.