The Power Of Sentiment Coupled With An Understanding Of Psychology -- NOTEBOOK ONLY

I've been trying to post some simple stuff on sentiment data to get people into it as I think it is easily the most powerful tool available.

This is another one of those posts showing just how powerful it can be.

This algorithm uses PsychSignal, Sentdex, and a series of sentiment data sets (CSV files)

no, I will NOT post the code.

8
Loading notebook preview...
Notebook previews are currently unavailable.
6 responses

Looks very stable and impressive it's long only! Jacob, can tell us more about the "Understanding Of Psychology" part?

Amazing. Thanks for posting this stuff. It's still a bit over my head how to get a useful signal from the noise so to speak, but this makes it look like an incredibly worthwhile endeavor.

It's funny how sharpe ratio penalizes sudden positive jumps in returns like that.

Is this using Q1500US? I've managed to get that kind of alpha while not using Q1500US, but my results are fairly dubious due to Q's backtester not accounting for bid-ask spread.

Why does it start performing worse with higher amounts of capital? Is it slippage due to more capital per fixed amount of stocks? Or are you having to ease up the pipeline screen to let lower quality stocks through in order to have enough capacity?

@Jacob, truly impressive results. Great work.

@Viridian Hawk, I believe quantopian accounts for bid-ask spread with their slippage model - which was on during all backtests. Either way, placing a limit order would "subvert" that problem - you'd just have a lot of open orders. This algorithm closes all orders before each transaction-causing function.

@Peter Bakker. I'll be making a post on this, but to answer your question - Sentiment data, as useful as it is, is flawed. Why? because humans are involved. Sentiment sourced from media is plagued (quite significantly) by many things. One of the most important is something that lies within all media, regardless of the language: Positivity Bias (known as Pollyannism) - the tendency for people to remember events in a more positive light (this happens at the subconscious level). Equally important is negativity bias which happens at the conscious level.

When a person says they think a stock will perform well, we must be concerned about their previous experience with that stock... obviously we can't look at this with the data available - We can't even determine if this person is just saying something just to say something...

To get around this, we need to get into sociology, to keep this post short, I am mostly concerned with the consequences of the hive mind (or herd mentality), and information loss (lack of fact checking).

Q1500? Nope, the entire universe of stocks.

@Jacob -- I'm delving deeper and deeper into the bid-ask spread issue, trying to understand it. The slippage model does have an alternative option to account for a fixed spread, but then it's fixed across all your securities and they give you no way of looking up what that spread might be. I was looking at bid-ask spreads last night and my mind was blown. A price might be $5, the bid might be$4.99 and the ask might be $10 -- lopsided and huge. Even the spread on Google was$1.50. I've heard multiple accounts from people who have gotten burned live trading that Q's slippage model is not accurate for stocks outside of the Q1500US. There are a lot of strategies that appear insanely profitable on low liquidity stocks, but I'm afraid the issue is that once bid-ask spread is accounted for, that inefficiency just isn't there. If anything limit orders might skew the results further from real life -- as the backtester might be giving you fills that didn't actually cross the spread. I'm waiting on Robinhood to confirm a second clean-slate account so I can just run some real-life tests and get a better sense of what the discrepancy is. So just keep this in mind, to at least be skeptical about results that include super-low volume stocks. Even if you don't want to limit to Q1500US, it might be worthwhile to add a screen for average daily volume above maybe 50k or something, and if that's hurting your returns one explanation is that those returns weren't realistic.

Anyways, that aside, thanks for your high level description of the psychology behind the algorithm. That's very clever if you're being able to find evidence of bias in sentiment and get a reliable trade signal from that. Congrats.

@Jacob, as you are using PsychSignal in this notebook, how were you able to get around the poor PsychSignal performance before 2015 that you mentioned in your other thread?