Back to Community
News API integration

Hi team,

I developed a real-time news aggregator and want to use the news articles returned by the API in my notebook. How can I make HTTP requests or create real-time channels using socket.io to connect to my API inside my notebook?

Web: https://newsfilter.io
Node.js libaray: https://www.npmjs.com/package/realtime-newsapi

6 responses

Quantopian doesn't allow web requests of any form for security reasons. However, a way you could still include your aggregator information is to use a self-serve dataset. You'll need your own code (offline) to output a .csv file that's compliant, but that isn't a huge technical challenge given the skillset you've shown.

I haven't done it myself yet, but there's a way to enable nightly updates of the self-serve dataset. In other words, if you use a static URL to an updating .csv file, Quantopian will pull it nightly. Someone please correct me if I'm wrong.

https://www.quantopian.com/posts/upload-your-custom-datasets-and-signals-with-self-serve-data

Very cool and exciting - thanks for sharing the links Jan!

Thanks @Kyle M! It seems I can only run the import of new news on a daily basis.

Optional live update files will be downloaded each trading day, between 07 to 10 am UTC

I wouldn't be able to react to M&A announcements, FDA approvals, or new SEC filings fast enough.

@Jan,

Great work! Although real time implementation may not be achievable under current Q framework, I still think that the content information can still provide alpha under daily frequencies. By adding a pre-processing layer that would transform these real time inputs into factors that would be clean and ingestible to the Q backtest and research APIs. Essentially, it is a one day delay information.

I'm here mostly to see the updates on this. I'm particularly interested to see if something can be factor inputted like James above said. I think everyone would be different in how they implement those factors, and seems rather labor intensive to create. I foresee a multi-year project, but one that if done well, could be very lucrative in a sense. Then to take it a step further and input into an ML algo.

Quantopian is interested in low turnover strategies where it doesn't matter whether you enter into your positions right now, later today, tomorrow, or the next day. So I don't see them adding any intraday data functionality. It's not too hard to get ahold of some intraday price/quote dataset and test your idea locally though. You might also find some lower frequency alpha that is usable on Quantopian -- such as if you can detect when the market has over-reacted to news.