Fetching Data from Disk

To retrieve external data stored on a local disk. use: fetch_external_data_from_disk(spec::SpecificationStruct{C,T}, file_path::String, datadir::String; clean_and_prepare=false) where {C<:AssetConstituents, T<:ExternalSpecification}

  • ext_cons: Specifies which external data constituents to fetch. See Constituents
  • file_path: The location on disk where the data is stored.
  • clean_and_prepare (optional, default is true): Whether to clean and prepare the data after retrieval.

This function supports both CSV and JLD2 file formats where a .csv file is used only as list of .jld2 files. The returned data can optionally be cleaned and prepared for further use.

Saving Data to Disk

You can save processed or fetched data onto your local disk using the save_file(edc::ExternalDataCollection, filename::String, datadir::String; overwrite=false)

  • edc: An instance of ExternalDataCollection that you intend to save. See ExternalDataCollection
  • filename: The desired name of the file where you wish to save the data. The full path will be designed by the resolution, see Resolution and Directory Structure
  • datadir: The root directory where you wish to save the data. The full path will be designed by the resolution, see Resolution and Directory Structure
  • overwrite (optional, default is false): Whether to overwrite an existing file with the same name.