Back to Community
Data Bundle Error - help

Hi Everyone,

I am trying to build a new data bundle for Thai Stock market to be used in back test on Zipline.
I register my stocks in .zipline/

Then, I create a new file in zipline/data/bundles/ to support the new data bundle which is called 'csv'. This will read local CSV files with 'OHLC' format like Yahoo data.

Finally, I ran 'zipline ingest -b csv', but I got the following error:

Could you please guide me how to fix the issue? Is it related with the number of lines in CSV files for each stock?

Thank you for your help.

[4833 rows x 6 columns])]

Now calling daily_bar_writer
Traceback (most recent call last):
File "/usr/local/bin/zipline", line 11, in
File "/usr/local/lib/python2.7/dist-packages/click/", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/", line 1066, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click/", line 895, in invoke
return ctx.invoke(self.callback, *ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/", line 535, in invoke
return callback(
args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/zipline/", line 312, in ingest
File "/usr/local/lib/python2.7/dist-packages/zipline/data/bundles/", line 451, in ingest
pth.data_path([name, timestr], environ=environ),
File "/usr/local/lib/python2.7/dist-packages/zipline/data/bundles/", line 106, in ingest
daily_bar_writer.write(liData, show_progress=False)
File "/usr/local/lib/python2.7/dist-packages/zipline/data/", line 257, in write
return self._write_internal(it, assets)
File "/usr/local/lib/python2.7/dist-packages/zipline/data/", line 378, in _write_internal
AssertionError: Got 3068 rows for daily bars table with first day=2002-08-09, last day=2017-07-07, expected 3754 rows.
Missing sessions: [Timestamp('2002-08-12 00:00:00+0000', tz='UTC'), Timestamp('2002-08-26 00:00:00+0000', tz='UTC'), Timestamp('2002-10-23 00:00:00+0000', tz='UTC'), Timestamp('2002-12-05 00:00:00+0000', tz='UTC'), Timestamp('2002-12-10 00:00:00+0000', tz='UTC'), Timestamp('2002-12-30 00:00:00+0000', tz='UTC'), Timestamp('2002-12-31 00:00:00+0000', tz='UTC'), Timestamp('2003-04-07 00:00:00+0000', tz='UTC'), Timestamp('2003-04-14 00:00:00+0000', tz='UTC'), Timestamp('2003-04-15 00:00:00+0000', tz='UTC'), Timestamp('2003-05-01 00:00:00+0000', tz='UTC'), Timestamp('2003-05-05 00:00:00+0000', tz='UTC'), Timestamp('2003-05-09 00:00:00+0000', tz='UTC'), Timestamp('2003-05-15 00:00:00+0000', tz='UTC'), Timestamp('2003-07-01 00:00:00+0000', tz='UTC'), Timestamp('2003-07-14 00:00:00+0000', tz='UTC'), Timestamp('2003-08-12 00:00:00+0000', tz='UTC'), Timestamp('2003-10-23 00:00:00+0000', tz='UTC'), Timestamp('2003-12-05 00:00:00+0000', tz='UTC'), Timestamp('2003-12-10 00:00:00+0000', tz='UTC'), Timestamp('2003-12-18 00:00:00+0000', tz='UTC'), Timestamp('2004-01-02 00:00:00+0000', tz='UTC'), Timestamp('2004-01-28 00:00:00+0000', tz='UTC'), Timestamp(................

3 responses

hi, did you solve your problem?

Not yet.

Thanks to Freddie and Delaney from Quantopain for help me out.

@Stephane, I hope you can follow the guides provided and sort your issues out.

Freddie suggested me as follows:

Zipline only supports US Equities (and broad support for futures), I'm not sure if the trading calendar associated with the Thai Stock Market will have all of the dates that Zipline expects from the NYSE Calendar.

There is currently a Pull Request open that implements a bundle that loads data from csv files into Zipline. You may want to use that branch that the person had made or compare your bundle to theirs in terms of implementation

If there are dates missing, in that your stock data has dates that the Zipline NYSE calendar does not have, then the best option I can see is that you backfill on those dates. There is another Pull Request that is not currently being worked on, but is open, that allows users to switch to different calendars. If there were a calendar you needed that was not there, you'd have to implement it using the Trading Calendar API. In regards to fixing the calendar issue, it looks like someone figured out a workaround for this issue in Zipline