Back to Community
New Dataset: Global Equity Metadata

The datasets described herein are proprietary to FactSet Research Systems, Inc. ("Factset") and may not be copied or distributed. The datasets made available to Quantopian by FactSet are not exhaustive of FactSet's data, products, software, and/or services.

Equity Metadata

Today, we added a new dataset to the platform: Equity Metadata. Equity Metadata is a dataset that includes information about equities such as the security type, listing currency, exchange, and more. The dataset combines fields from FactSet Symbology and FactSet Fundamentals that are commonly used to build a tradable universe.

Contest algorithms are still required to trade within the QTradableStocksUS universe, but non-US pipelines don't have an equivalent definition. The new Equity Metadata dataset allows you to make a tradable universe in non-US markets so that you can construct factors on a liquid set of equities with similar properties. Unfortunately, the contest doesn't yet support non-US markets, but we are working toward that goal.

Equity Metadata is available via the Pipeline API, which means it can be accessed in Research and the IDE.


Properties

  • Coverage: All supported countries on Quantopian
  • Data Frequency: Irregular/static; most equity metadata is expected to be static or only change infrequently due to some sort of corporate action/event
  • Update Frequency: Daily (updated overnight after each trading day)
  • Timespan: North American equities are covered from 2004 to present; other region start dates can be found here
  • Holdout: 1 year

Data Holdout Period

Like other FactSet-sourced datasets, Equity Metadata has a holdout period. In this case, Equity Metadata has a trailing 1-year holdout. This means that the most recent year of data is not accessible in Research and Backtesting. However, submitting an algorithm to the contest that uses Equity Metadata is allowed and all contest scoring is done using the entire, up-to-date dataset. Similarly, algorithms using Equity Metadata will be evaluated by Quantopian for funding using the full dataset.

Point-In-Time

Starting in June 2018, Equity Metadata is collected and surfaced in a point-in-time fashion on Quantopian. This corresponds to when Quantopian started downloading and storing Equity Metadata on a nightly basis.


Usage

The EquityMetadata dataset is a pipeline DataSet. The columns of the EquityMetadata dataset can be used like any other BoundColumn in a pipeline.

Example

This code snippet constructs and runs a pipeline of Canadian equities that gets screened down to a tradable universe. The tradable universe is defined as the top 50% of equities (ranked by market cap) of security type SHARE that are also the primary share of the company. Make sure to run it in Research.

from quantopian.pipeline import Pipeline  
from quantopian.pipeline.data.factset import EquityMetadata, Fundamentals  
from quantopian.pipeline.factors import Returns  
from quantopian.pipeline.domain import CA_EQUITIES  
from quantopian.research import run_pipeline

# Create a factor of latest market cap.  
mcap = Fundamentals.mkt_val.latest

# Create a pipeline filter for 'tradable' stocks.  
is_tradable = (  
    EquityMetadata.security_type.latest.eq('SHARE')  
    & EquityMetadata.is_primary.latest  
)

# Create a base universe filter that selects the top 50% of our 'tradable'  
# equities based on market cap.  
base_universe = mcap.percentile_between(50, 100, mask=is_tradable)

# Create a trailing 1-week returns factor and a latest quarterly sales factor.  
returns_1w = Returns(window_length=6)  
latest_sales = Fundamentals.sales_qf.latest

# Build a pipeline with our returns and quarterly sales factors. The pipeline is defined  
# over the Canadian equities domain and gets screened down to our base universe.  
pipe = Pipeline(  
    columns={  
        'quarterly_sales': latest_sales,  
        'returns': returns_1w,  
    },  
    domain=CA_EQUITIES,  
    screen=base_universe,  
)

# Execute our pipeline and print the first few rows of the output DataFrame.  
df = run_pipeline(pipe, '2015-05-05', '2016-05-05')  
df.head()  

The attached notebook provides a similar example as well as some analysis of the dataset.


Pipeline Datasets and Columns

Dataset

EquityMetadata - The EquityMetadata dataset is a pipeline dataset that provides access to equity information most commonly used for constructing a tradable universe.

Fields

The EquityMetadata dataset has 5 fields (accessible as BoundColumn attributes):

  • security_type (dtype str): A string code representing the security type of the equity. The table below lists and describes the set of possible security types.
  • is_primary (dtype bool): Boolean flag denoting whether this listing is the primary issue of the security (True = primary issue, False = secondary issue).
  • listing_currency (dtype str): Currency in which the listing trades.
  • listing_exchange (dtype str): Code denoting the exchange on which the listing trades. See list of supported exchanges here.
  • primary_fsym_security_id (dtype str): The primary FactSet security ID of the equity. Note that multiple equities can have the same FactSet security ID as a particular security can have multiple listings across different regions.

Security Types

Code Short Description Long Description
ADR American Depositary Receipt An ADR is issued by a US depositary bank and represents shares of a foreign corporation held by the bank. ADRs provide foreign companies with an access to US capital markets. The instruments typically trade in US dollars.
ALIEN Alien/Foreign Shares that foreign investors can purchase from countries where they issue shares, such as Thailand. Foreigners can register these shares under their own name using the Foreign Board and receive all benefits offered by the company, such as rights offerings and dividends. Such securities have no floor or ceiling limits, so share prices respond to the demand and supply.
DR Depositary Receipts/Certificates This security type represents all other depository receipts not mentioned above. Depositary Receipts (DR) are negotiable certificates that enable domestic investors to own shares in foreign companies. Each DR represents a given number of a company's shares.
ETF_ETF Exchange Traded Fund (ETF-ETF) A security that is created to track the performance of an index or a basket of assets. ETFs are traded on organized exchanges.
GDR Global Depositary Receipt Certificate that represents ownership of a given number of a company's shares and can be listed and traded independently from the underlying shares. Provide issuers with an access to foreign capital markets.
MF_C Closed-ended Fund These funds have a fixed number of shares, which are listed on the stock exchange. The market price of the shares is determined by demand and supply factors. These are readily transferable in the open market and are bought and sold like other stock. These funds do not accept new contributions from investors, but only reinvest the return on the existing portfolio.
MF_O Open-ended Fund These funds have no limit to the number of units (or shares) they can issue. The price of the units (or shares) remains closely aligned to the NAV of the fund. Open-end funds sell their own new shares to investors, stand ready to buy back their old shares, and are not listed on exchanges. Open-end funds are so called because their capitalization is not fixed.
NVDR Non-Voting Depositary Receipt A Non-Voting Depository Receipt (NVDR) is a trading instrument issued by the Thai NVDR Company Limited, a subsidiary wholly owned by the Stock Exchange of Thailand (SET). Investment in NVDR eliminates foreign investment barriers. The only difference between investing in NVDRs and the company's shares is the voting right. NVDRs are automatically regarded as listed securities in the SET.
PREF Preferred A type of stock that pays a fixed dividend regardless of corporate earnings, and which has priority over common stock in the payment of dividends. The main characteristics of preferred stocks is the fixed income stream and priority of payment at liquidation.
PREFEQ Preferred Equity A security that has equity characteristics (i.e., ownership at liquidation, share in the profits in the company) rather than fixed redemption and trades as a common equity (i.e., is very liquid and its prices are quoted in a currency rather than percentage of par, which is a common characteristic of preferred stock). These securities are most commonly issued in Brazil, Canada, and many European markets (i.e., Russia, Germany, and Italy). Preferred equity has some priority rights over common stock. This category also includes Italian savings shares.
RIGHT Right A right gives the holder an option to purchase additional shares of an underlying instrument at a pre-determined price on a specified date (typically at a discount from the market price). In most cases, right instruments must be exercised within a relatively short period.
SHARE Share/Common/Ordinary Securities that represent an ownership interest in a corporation. If the company has also issued preferred stock, both common and preferred stockholders have ownership rights. Common stockholders assume the greater risk, but generally exercise greater control and may gain a larger award in the form of dividends and capital appreciation.
STRUCT Structured Products Investment products that are based on an underlying security such as a single equity, a basket of stocks, an index, a commodity, a debt issuance, or a foreign currency. Structured products allow investors to buy a single unit/share of a fund that represents an interest in the investment portfolio.
TEMP Temporary Securities that trade for a limited period of time (typically less than a year) and trade alongside the mainline security. Also included are securities that trade alongside the mainline and then assimilate to the mainline security after trading for few days/months. An example of temporary instruments are parallel lines and assented lines.
UNIT Unit An equity that consists of more than one equity instrument (i.e., security consisting of class A and B; 1 share and 1 warrant).
WARRANT Warrant An instrument giving the holder a right to subscribe to an underlying instrument at a given price and date.

Other Notes

  • Currently, non-US data is only available in Pipeline in Research. The IDE only has access to US equity data at this time.
  • This is currently the best documentation for the Equity Metadata dataset. We are working on a new set of documentation that will document all data integrations in one place.
Loading notebook preview...
Notebook previews are currently unavailable.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

3 responses

Excellent, Jamie with thanks!

Great - thank you!

Hi Jamie,

As Trading Algorithm is not supported in Research, it seems not that much help for the global equity metadata.
When will you support the other domains (i.e., HK_EQUITIES) in pipelines in IDE/Backtester?

Thanks!