Back to Community
How to utilize FRED datasets in pipeline?


New to the platform, but having a hard time figuring out how to use some of the datasets, for example:

from import rateinf_inflation_usa # Inflation rate  
from import fred_gnp # GNP  
from import fred_gdp # GDP  
from import fred_icsa # Unemployment Claims  
from import adp_empl_sec # Total Jobs  

These datasets are in the form of a Blaze expr. object. I figured out how to make them a Pandas.dataFrame, but I have no idea how to access the nearest time values (asof_date) in a CustomFactor or in the Pipeline itself.

What I would like to do is this:

For every day, look at the % change for each of the above values (GDP, GNP, etc) between the current value and the value a month ago).

Would appreciate any help on this, as I've been unable to make much progress. Thank you.

2 responses

Currently handling with an (inefficient) CustomFactor. Here I am iterating over each entry to find the closest date. Would love a more efficient approach, though.

from import adp_empl_sec 

jobs = blaze.compute(adp_empl_sec)  
jobs = jobs.sort_values("asof_date")

def find_most_recent_jobs(date):  
    for i in range(len(jobs)):  
        asof_date = jobs.iloc[i].asof_date  
        date = date.replace(tzinfo=None)  
        if asof_date > date:  
            latest = jobs.iloc[i-1].total_private  
            prev = jobs.iloc[i-2].total_private  
            pct_change = latest / prev - 1  
            return pct_change

class TrainingData(CustomFactor):  
    inputs = []  
    outputs = ["jobs_change", "gnp_change", "gdp_change", "inflation_change", "unemployment_change"]  
    window_length = 30  
    def compute(self, today, assets, out):  
        jobs_change = find_most_recent_jobs(today)  
        out.jobs_change[:] = jobs_change

Hi Thomas,

Here's a notebook that gets the percent change between the last two Fred GNP values. I also put in some custom factors that get the latest GNP value as well as the latest corresponding asof_date so that you can play around with it. Note that since this is a value that updates quarterly, the percent_change will be forward filled on most days.

Also, Pipeline is limited to outputting dataframes where the result is mapped to US equities. To work with macroeconomic data, I broadcast the result across all of the assets and added a StaticAssets filter to the pipeline to just pull up GNP value and percent change each day.

Let me know if this helps.

Loading notebook preview...

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.