Skip to content

Latest commit

 

History

History
49 lines (34 loc) · 1.97 KB

_add-data-twelvedata-crypto.md

File metadata and controls

49 lines (34 loc) · 1.97 KB

Load financial data

This tutorial uses real-time cryptocurrency data, also known as tick data, from Twelve Data. To ingest data into the tables that you created, you need to download the dataset, then upload the data to your $SERVICE_LONG.

  1. Unzip crypto_sample.zip to a <local folder>.

    This test dataset contains second-by-second trade data for the most-traded crypto-assets and a regular table of asset symbols and company names.

    To import up to 100GB of data directly from your current $PG-based database, migrate with downtime using native $PG tooling. To seamlessly import 100GB-10TB+ of data, use the live migration tooling supplied by $COMPANY. To add data from non-$PG data sources, see Import and ingest data.

  2. In Terminal, navigate to <local folder> and connect to your $SERVICE_SHORT.

    psql -d "postgres://<username>:<password>@<host>:<port>/<database-name>"

    The connection information for a $SERVICE_SHORT is available in the file you downloaded when you created it.

  3. At the psql prompt, use the COPY command to transfer data into your $SERVICE_LONG. If the .csv files aren't in your current directory, specify the file paths in these commands:

    \COPY crypto_ticks FROM 'tutorial_sample_tick.csv' CSV HEADER;
    \COPY crypto_assets FROM 'tutorial_sample_assets.csv' CSV HEADER;

    Because there are millions of rows of data, the COPY process could take a few minutes depending on your internet connection and local client resources.