Back to Community
News API integration

Hi team,

I developed a real-time news aggregator and want to use the news articles returned by the API in my notebook. How can I make HTTP requests or create real-time channels using socket.io to connect to my API inside my notebook?

Web: https://newsfilter.io
Node.js libaray: https://www.npmjs.com/package/realtime-newsapi

5 responses

Quantopian doesn't allow web requests of any form for security reasons. However, a way you could still include your aggregator information is to use a self-serve dataset. You'll need your own code (offline) to output a .csv file that's compliant, but that isn't a huge technical challenge given the skillset you've shown.

I haven't done it myself yet, but there's a way to enable nightly updates of the self-serve dataset. In other words, if you use a static URL to an updating .csv file, Quantopian will pull it nightly. Someone please correct me if I'm wrong.

https://www.quantopian.com/posts/upload-your-custom-datasets-and-signals-with-self-serve-data

Very cool and exciting - thanks for sharing the links Jan!

Thanks @Kyle M! It seems I can only run the import of new news on a daily basis.

Optional live update files will be downloaded each trading day, between 07 to 10 am UTC

I wouldn't be able to react to M&A announcements, FDA approvals, or new SEC filings fast enough.

@Jan,

Great work! Although real time implementation may not be achievable under current Q framework, I still think that the content information can still provide alpha under daily frequencies. By adding a pre-processing layer that would transform these real time inputs into factors that would be clean and ingestible to the Q backtest and research APIs. Essentially, it is a one day delay information.

I'm here mostly to see the updates on this. I'm particularly interested to see if something can be factor inputted like James above said. I think everyone would be different in how they implement those factors, and seems rather labor intensive to create. I foresee a multi-year project, but one that if done well, could be very lucrative in a sense. Then to take it a step further and input into an ML algo.