Today, we added a new Custom Dataset Dashboard to help you create and monitor the datasets that you upload to Quantopian using Self-Serve Data. Since the Self-Serve Data introduction just over a year ago, thousands of custom datasets have been added to the platform, several of which have been used in contest algorithms and fund algorithms.
With increased usage came a need to improve the workflow around uploading and managing custom datasets. The new Custom Dataset Dashboard was added to make it easier to work with custom datasets on Quantopian. If you are not yet familiar with Self-Serve Data and you would like to upload a custom dataset to Quantopian, check out the documentation here.
- Search: Filter datasets by searching all the column data for entered search text.
- Column sort order: By default the datasets are sorted by Created At in descending order. You can select any column including name to reorder the datasets table.
- historical: Single historical load only, no live updates configured.
- live: Dataset is configured for live updates.
- archived: Dataset with disabled live updates, may contain point-in-time data. Note that archiving is not yet available, but should be available soon (see bottom of this post).
- error: The initial dataset failed during the historical load, import functionality is available but the data typically contains 0 rows.
Hover over the status value to see the last load timestamp and error message if status = failed.
- pending: a brand new dataset that has been added to the queue, but hasn't started running yet
- running: an in-process load
- completed: a successful load, may result in zero updated rows during each live update check
- failed: the load did not succeed. Use hover to see error message
- empty: No data records were found during live update
The timestamp when the dataset first was created in the local timezone and locale date format.
Note: this will be blank for error type datasets.
Coming soon: Delete and Archive (disable live updates) functionality plus an additional column to help monitor the Last Updated Date timestamp for live and archived datasets.
Note: for a short transition period, the old Self-Serve Data page will continue to function. In the near future, we will redirect traffic to the new dashboard.