Back to Community
Global Equity Pricing and Fundamental Data

Data Available for 25 Countries

Today, we added the first global equity data to Quantopian. Global equity pricing data and fundamentals for 25 countries are now accessible via the Pipeline API in Research. The table below lists the newly supported countries as well as the corresponding supported exchanges.

Country ISO Alpha-2 Code Supported Exchanges
Austria AT Vienna Stock Exchange
Australia AU Australian Securities Exchange, National Stock Exchange of Australia
Belgium BE Euronext Brussels
Brazil BR Sao Paulo Stock Exchange
Canada CA Toronto Stock Exchange, TSX Venture Exchange, Canadian Securities Exchange
China CN Shenzhen Stock Exchange, Shanghai Stock Exchange
Denmark DK NASDAQ OMX Copenhagen
Finland FI NASDAQ OMX Helsinki
France FR Euronext Paris
Germany DE Berlin Stock Exchange, Dusseldorf Stock Exchange, XETRA, Frankfurt Stock Exchange, Hamburg Stock Exchange, Hannover Stock Exchange, Munich Stock Exchange, Stuttgart Stock Exchange, Xetra Indices
Great Britain GB London Stock Exchange, ICAP Securities & Derivatives Exchange, Cboe Europe Equities CXE
Hong Kong HK Hong Kong Stock Exchange
India IN Bombay Stock Exchange, National Stock Exchange of India
Ireland IE Irish Stock Exchange, Irish Stock Exchange Bonds & Funds
Italy IT Milan Stock Exchange
Japan JP Tokyo Stock Exchange, JASDAQ, Osaka Exchange, Nagoya Stock Exchange, Fukuoka Stock Exchange, Sapporo Securities Exchange
Netherlands NL Euronext Amsterdam
New Zealand NZ New Zealand Stock Exchange
Norway NO Oslo Exchange
Portugal PT Euronext Lisbon
Singapore SG Singapore Exchange
Spain ES Madrid Stock Exchange/Spanish Markets
Sweden SE NASDAQ OMX Stockholm, AktieTorget, Nordic Growth Market
Switzerland CH SIX Swiss Exchange, BX Swiss AG, Swiss Fund Data
United States US NYSE, NASDAQ, AMEX

 

Pricing and fundamentals data for each of these countries is available as far back as 2004. The pricing data is daily, including OHLCV bars. Global fundamentals are sourced from the recently announced FactSet Fundamentals data integration. All global equity data is accessed using a new Pipeline feature, domains, described below and in the attached notebook.


International Pipelines & Domains

Pipeline is a tool that allows you to define computations over a universe of assets and a period of time. In the past, you could only run pipelines on the US equity market. As of today, you can now specify a domain over which a pipeline should be computed. The name "domain" refers to the mathematical concept of the "domain of a function", which is the set of potential inputs to a function. In the context of Pipeline, the domain specifies the set of assets and a corresponding trading calendar over which the expressions of a pipeline should be computed.

Currently, there are 25 domains available in the Pipeline API corresponding to the countries in the table above. The attached notebook explains pipeline domains in more detail.

Example Usage

The following pipeline returns the latest close price, volume, and market cap for all Canadian equities, every day. Make sure to run it in Research.

from quantopian.pipeline import Pipeline  
from quantopian.pipeline.data import EquityPricing, factset  
from quantopian.pipeline.domain import CA_EQUITIES  
from quantopian.research import run_pipeline

pipe = Pipeline(  
    columns={  
        'price': EquityPricing.close.latest,  
        'volume': EquityPricing.volume.latest,  
        'mcap': factset.Fundamentals.mkt_val.latest,  
    },  
    domain=CA_EQUITIES,  
)

result = run_pipeline(pipe, '2015-01-15', '2016-01-15')  
result.head()  

To learn more about domains, see the attached notebook. If you're curious, you can read more about the design behind domains in this Zipline issue.


Currency Conversions

One of the challenges in working with international financial data is the fact that prices and price based fields can be denominated in different currencies. Even if you only want to research stocks listed on a single exchange, currencies can present a challenge. For example, a company that is based in the US (like Apple), reports their financials in USD, but lists shares on exchanges around the world. These listings are often denominated in the local currency of the exchange. Having pricing and fundamentals data denominated in different currencies makes it hard to make comparisons between the two datasets.

To solve this problem, FactSet Fundamentals data is denominated in the listing currency of each asset. For example, if you query sales data for the US listing of Apple, you will get data denominated in USD. If you look up fundamentals for the Japanese listing of Apple (listed in Japanese Yen), you will get data from the same US report, but it will be converted to Japanese Yen. The specific conversion techniques are as follows:

Income Statement and Cash Flow Statement

Data from the income statement and cash flow statement of a fundamentals report is converted using the average exchange rate over the fiscal period in question.

Balance Sheet

Data from the balance sheet of a report is converted using the exchange rate from the end of the fiscal period of the report (i.e. the end of Q1 for a Q1 quarterly report).


Other Notes

Contest Eligibility

Unfortunately, international markets are not yet supported in the backtester, and by extension, they are not yet supported in the contest. We don’t currently have a timeline for a contest that supports international markets. We will provide an update to the community when we have more information to share.

Alphalens

Alphalens is currently the best tool for analyzing a factor on an international market. To date, we have suggested that you use get_pricing to get daily pricing data and supply it to Alphalens. International pricing data isn't yet available in get_pricing, which motivated us to revisit this suggestion. After poking around a bit, we put together a wrapper function that makes it easy to write a pipeline factor and run it through alphalens without having to write all of the get_pricing boilerplate code. The notebook attached in the comment below includes the wrapper function evaluate_factor. You can use evaluate_factor to evaluate a factor on any domain.

Holdouts

As described in this post, the most recent year of FactSet Fundamentals data is held out from the community platform, which applies to both US and international data. The same holdout applies to the daily OHLCV data for non-US markets. This means that all research on international factors will need to be conducted on data that is more than a year old. We are hopeful that the holdout will actually help to reduce the risk of overfitting.

Documentation

The notebook in this post is the best reference material on domains at this time. We plan to add documentation and other learning material around domains in the near term.

Multi-Currency Markets

Another challenge related to currencies is the fact that some exchanges don't require stocks to be listed in local currency. For example, the London Stock Exchange only has about 75% of its listings denominated in GBP*. The other 25% are primarily listed in EUR or USD. This can make it hard to make cross sectional comparisons.

To solve this problem, most people rely on currency conversions to bring price-based fields into the same currency. Currently, there's no way to convert pricing and fundamentals data, so everything is denominated in the listing currency (the default). In addition, the listing currency information isn't yet accessible, so you don't have an easy way to determine the currency of a particular data point. Our long-term plan is to solve this problem and make it easy to conduct analyses across assets listed in different currencies.

In the meantime, currency-dependent fundamentals fields are not yet available for assets that are listed in a non-primary currency of their exchange. Pricing data for non-primary currency assets is available, but you should use returns instead of raw prices to build a factor on a non-US domain (probably a good idea anyway!). Working in returns space instead of pricing makes it reasonable to make comparisons across assets listed in different currencies.

*Great Britain, Switzerland, and Ireland all have a significant number of assets listed in non-local currency. Other markets have the vast majority, if not all of their listings in local currency.

Combining Domains

Pipelines can now be defined with new domains, but the idea of 'combining' domains is not yet supported. When working with international equity data, it is a common use case to develop a factor at a regional level (for example, Europe) instead of a country level. We plan to support multi-country domains, but don't yet have a timeline on the delivery of this feature.

Known Bugs
(We will make an effort to update this list as we discover and fix bugs)
- Fixed: Fundamentals data for Sweden, Denmark, and Ireland are not yet available. We expect these to be available soon.* (Sweden and Denmark fundamentals data is now available)
- Fundamentals data for Ireland is not yet available.
- Starting a non-US pipeline on a non-trading day (in the local market) raises an exception.


Try building a factor on a new domain and let us know what you think. As always, if you discover any issues that aren't listed above or if you have any questions about the new data, please let us know!

Loading notebook preview...
Notebook previews are currently unavailable.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

24 responses

Here's a notebook that builds a factor in an international market and tests it with Alphalens. Note the use of a new helper function, evaluate_factor, to prepare the factor and returns data for use in Alphalens.

Loading notebook preview...
Notebook previews are currently unavailable.

Game changer! Christmas comes early this year!

Can’t believe you support AktieTorget and Nordic Growth Market (NGM) even. Sooooo cool!

Thank you for the hard work. The Alphalens example for international markets is greatly appreciated.

This is great! I've been looking for a way to test Canadian equities for a while now. I've noticed a lot of missing fundamental data for some of the smaller Canadian stocks. Are there restrictions on data for some securities or is it just that this is currently being worked on?

Hi Jason,

Can you share the list of SIDs for which you are seeing missing fundamental data? I'd be happy to look into it. Feel free to email the list to [email protected] with the details.

Will do!

Updates:
- Fundamentals data for Sweden (SE_EQUITIES) and Denmark (DK_EQUITIES) are now available.
- Pricing and Fundamentals data are now available for Brazil (BR_EQUITIES).

@Jamie McCorriston

I've run into a bit of a problem trying to gather all of the SID's with missing data. I can't export data and the list is too long to print.
Equity(1178883448845123 [WRA]) NaN
Equity(1178883868150594 [KAR]) NaN
Equity(1178887761843025 [IES]) NaN
Equity(1178888030997576 [GOR]) NaN
Equity(1178892107470674 [ISN])

That was the print from the .head() of securities with a Nan market cap in Canada. Not sure how to send over the entire list.

Great work! Looking forward to using global FactSet in algorithm API.

Hi @Jamie,

I’m curious if you’d be willing to share a high level roadmap (with or without rough timelines) of what the plan is with Factset data and global domains in the near future (or longer term)?

For example, I’m guessing you might plan to work out the challenges with currency conversions and combining domains (and fix known bugs) before you plan to support new datasets or global domains in the IDE?

Update:
- Added pricing and fundamentals data for China (CN_EQUITIES), India (IN_EQUITIES), and Singapore (SG_EQUITIES).

Hi @Jamie McCorriston,
I have few queries regarding Indian Equities data:
1. Is it NSE or BSE data?
2. Is FnO data also available?

Hi Jamie,
I was wondering if there is an easy way to combine multiple countries?
Pieter-Jan

@Adhish: NSE and BSE data is included. The original post on this thread lists the supported exchanges for each market. Regarding FnO data, is that short for 'Futures & Options'? If so, the answer is no. We currently support equity pricing and fundamentals data.

@Pieter-Jan: Right now, there isn't an easy way to combine domains/countries. If you want, you can merge the output of pipeline results from different countries, but it will take some manual effort. If you want to merge the outputs, remember that all price-based data is denominated in local currency, so you should try to move any price-based factors to returns space before combining them.

Thank you so much for this! this is amazing!
Do you see any of these exchanges being integrated with the backtester in the coming year?

Hi @Jamie,

Thank you for working on this project.
I believe there is a typo error in the example code:
'volume': EquityPricing.close.latest, should be 'volume': EquityPricing.volume.latest

Regards,
Apoorv

Are the equity prices adjusted for splits, dividends etc ?

Also, to avoid look-ahead bias, is there a FactSet field that records when the quarterly reports were actually released ?

@Boris: We plan to add support for global equity trading in the backtester, but we don't have a timeline as to when it will be available.

@Apoorv: Thanks for pointing out the typo. I just edited the post to correct it.

@Ma: Yes, prices are adjusted for splits and dividends just like our US data. They will be adjusted point-in-time in Pipeline. There's an explanation here that you might find helpful. Regarding fundamentals data, FactSet provides a report date for most records. When a record has a report date, it is surfaced the following day in Pipeline. In cases where we don't get a report date, the report is artificially delayed based on the reporting criteria of the country in which it trades. For instance, in the US, quarterly reports can be filed up to 45 days after the end of the fiscal period to which the report corresponds. Annual reports can be published up to 60 calendar days after the fiscal year end. FactSet fundamental data in Pipeline is lagged accordingly when the report date is not available.

Does this help?

Hi I'm a quantopian member but I know very little about it. At the aame time, my question is very straightforward. It is possible to construct a strategy that requires intraday prices for brazilian stocks and trades through the bovespa exchange. Thanks for any insights.

                                                                                                                                                                                                                        Mark  

Hi,
Are there plans for minute-level data for global markets in the near future?
Thanks

When will 1-minute data of NSE India come ? Any idea please....

@mark, @Rajkumar, @Rahul: We do not currently support minute-level international data and we do not have a timeline as to when it will be added to the platform. Sorry for the inconvenience.

thanks Jamie. It's appreciated.