External Data Handler - Functions

external_data_paths_from_csv(fname::String)

Given a CSV filename, extracts and returns the relative paths to data files specified in it. Ensures the input file has a CSV extension before processing.

fetch_external_data_from_disk(spec::SpecificationStruct{C,T}, file_path::String,
    datadir::String; clean_and_prepare=false) where {C<:AssetConstituents, T<:ExternalSpecification}

Retrieves external data from disk given a path. Optionally cleans and prepares the data after loading. Handles both CSV and JLD2 file formats where the CSV should be a file pointing to a list of JLD2 files.

load_data_from_disk(path::Vector{String}, datadir::String; tickers::Vector{String}=[])

Loads specific tickers' data from multiple disk paths. Aggregates the data from all paths into a single ExternalDataCollection.

load_data_from_disk(path::String, tickers::Vector{String})

Loads specific tickers' data from the provided path on the disk. Returns an ExternalDataCollection containing the loaded data.

resolution_to_dir(resolution::String)

Maps a data resolution string (like "1d" or "1m") to its respective directory path. Helpful for organizing data storage based on the resolution of the data.

save_file(edc::ExternalDataCollection, filename::String, datadir::String; overwrite=false)

Persistently saves the ExternalDataCollection to disk under the specified filename. If overwrite is set to true, it will replace any existing file with the same name.

fetch_external_data_from_internet(spec::SpecificationStruct{C,T}, service::Type;
clean_and_prepare=true, verbose_error=false, api_key="") where {C<:AssetConstituents,T<:ExternalSpecification}
                    clean_and_prepare=true, verbose_error=true ) where {C<:AssetConstituents, T<:ExternalSpecification}

Fetches external data for the given constituents ext_cons using the specified service. Optionally cleans and prepares the data after fetching.

• spec: A specification of which external data constituents and data fields to fetch.
• service: The internet data service to use (e.g., YahooData).
• clean_and_prepare: Whether or not to clean and prepare the data after fetching.
• verbose_error: If true, provide detailed error messages.

Returns the fetched (and optionally cleaned) external data.