Back to Community
QTradableStocksUS volume

There will be a reduction in the number of partial fills with QTradableStocksUS() and to take a look at that a bit I wound up with this code. It seeks to compare overall daily average volume in relation to the lower half of sorted volume values. The theory: There could possibly be spikes that might cause a security to be included while its volume could possibly be often very low. I would also like help with a factor to be able to control that difficult challenge more flexibly at any time, maybe removing highs before averaging?

Logging is the point in this backtest and output is included in the source. It's an effort to compare share volumes in Q1500US() vs QTradableStocksUS(). If someone has another way to go about it they might find this tracking of maximums and code useful.

Would like to ask a favor from someone comfortable in research, if you can chart maybe the sorted low-to-high mean of each day's volume for WMS and PG for 200 days from around 2017-06-22 since they are on opposite ends of the spectrum. (PG determined by also logging minimums).

WMS is an outlier with overall volume for a day divided by lower half volumes at 6641, such a high number that I would not be surprised if there is an error.
2017-06-22 12:59 track_values:216 INFO now 6641.1 avg 13.6 lo 1 hi 6641 new or near max WMS

Notice in the Source Code tab that the pasted logging output shows an average for QTradableStocksUS / Q1500US at 13.6 / 9.3 or 146%, I'm not sure what to make of that, maybe it could mean that QTradableStocksUS has higher highs in tandem with lower lows or it could just be a flaw in my logic.

Clone Algorithm
19
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
'''
Using Q1500US() and then QTradableStocksUS(), trying to validate that the latter has 
  higher volume in a way that would result in the likelihood of fewer partial fills.
  Are its stocks higher volume on the low end or just higher average overall?

  Running this for a little under 10 months should be near 200 trading days.

- - - - - - - - - - - - - - - - - - - - - - - - 
Q1500US()
- - - - - - - - - - - - - - - - - - - - - - - - 
        volume_clipped[s] = volume[s].sort_values()[:int(len(volume) * .5)].fillna(0)
2016-07-27 12:59 track_values:188 INFO now 3.4  avg 3.4  lo 3  hi 3  new or near max ARNC
2016-07-27 12:59 track_values:188 INFO now 5.2  avg 3.7  lo 3  hi 5  new or near max ABMD
2016-07-27 12:59 track_values:188 INFO now 6.5  avg 3.7  lo 3  hi 7  new or near max AES
2016-07-27 12:59 track_values:188 INFO now 7.0  avg 3.9  lo 3  hi 7  new or near max AGCO
2016-07-27 12:59 track_values:188 INFO now 6.9  avg 3.9  lo 2  hi 7  new or near max ALKS
2016-07-27 12:59 track_values:188 INFO now 6.9  avg 3.8  lo 2  hi 7  new or near max APOG
2016-07-27 12:59 track_values:188 INFO now 9.4  avg 4.0  lo 2  hi 9  new or near max ATU
2016-07-27 12:59 track_values:188 INFO now 8.5  avg 4.1  lo 2  hi 9  new or near max AAN
2016-07-27 12:59 track_values:188 INFO now 7.9  avg 4.2  lo 2  hi 8  new or near max ARW
2016-07-27 12:59 track_values:188 INFO now 8.3  avg 4.3  lo 2  hi 8  new or near max ASB
2016-07-27 12:59 track_values:188 INFO now 11.6  avg 4.5  lo 2  hi 12  new or near max AVP
2016-07-27 12:59 track_values:188 INFO now 13.1  avg 4.6  lo 2  hi 13  new or near max AZO
2016-07-27 12:59 track_values:188 INFO now 25.7  avg 5.0  lo 2  hi 26  new or near max CACC
2016-07-27 12:59 track_values:188 INFO now 26.4  avg 5.3  lo 2  hi 26  new or near max CBRL
2016-07-27 12:59 track_values:188 INFO now 27.6  avg 5.5  lo 2  hi 28  new or near max CLH
2016-07-27 12:59 track_values:188 INFO now 66.8  avg 5.9  lo 2  hi 67  new or near max COHR
2016-07-27 12:59 track_values:188 INFO now 116.0  avg 6.3  lo 2  hi 116  new or near max FUL
2016-07-27 12:59 track_values:188 INFO now 266.5  avg 6.9  lo 2  hi 266  new or near max ALR
2016-07-27 12:59 track_values:188 INFO now 276.7  avg 7.3  lo 2  hi 277  new or near max CSGP
2016-07-27 12:59 track_values:188 INFO now 288.2  avg 8.2  lo 2  hi 288  new or near max SCAI
2016-07-28 12:59 track_values:188 INFO now 378.5  avg 8.5  lo 2  hi 379  new or near max DNB
2016-07-28 12:59 track_values:188 INFO now 794.2  avg 9.1  lo 2  hi 794  new or near max HUBS
2016-08-19 12:59 track_values:188 INFO now 852.2  avg 8.9  lo 1  hi 852  new or near max SAAS
2016-08-31 12:59 track_values:188 INFO now 1299.5  avg 9.3  lo 1  hi 1299  new or near max SAAS
2016-09-06 12:59 track_values:188 INFO now 1243.3  avg 9.3  lo 1  hi 1243  new or near max FEIC
2016-09-08 12:59 track_values:188 INFO now 2804.8  avg 9.3  lo 1  hi 2805  new or near max OUTR

- - - - - - - - - - - - - - - - - - - - - - - - 
QTradableStocksUS()
- - - - - - - - - - - - - - - - - - - - - - - - 
        volume_clipped[s] = volume[s].sort_values()[:int(len(volume) * .5)].fillna(0)
2016-07-27 12:59 track_values:216 INFO now 3.4  avg 3.4  lo 3  hi 3  new or near max ARNC
2016-07-27 12:59 track_values:216 INFO now 9.9  avg 5.3  lo 3  hi 10  new or near max ABAX
2016-07-27 12:59 track_values:216 INFO now 13.9  avg 7.4  lo 3  hi 14  new or near max DDC
2016-07-27 12:59 track_values:216 INFO now 72.3  avg 20.4  lo 3  hi 72  new or near max ABM
2016-07-27 12:59 track_values:216 INFO now 112.4  avg 8.6  lo 2  hi 112  new or near max BGG
2016-07-27 12:59 track_values:216 INFO now 155.2  avg 10.6  lo 2  hi 155  new or near max CBU
2016-07-27 12:59 track_values:216 INFO now 142.6  avg 8.9  lo 2  hi 143  new or near max ITRI
2016-07-27 12:59 track_values:216 INFO now 266.5  avg 9.5  lo 2  hi 266  new or near max ALR
2016-07-27 12:59 track_values:216 INFO now 276.7  avg 9.8  lo 2  hi 277  new or near max CSGP
2016-07-27 12:59 track_values:216 INFO now 271.6  avg 10.0  lo 2  hi 272  new or near max NPO
2016-07-27 12:59 track_values:216 INFO now 244.7  avg 10.6  lo 2  hi 245  new or near max TUMI
2016-07-27 12:59 track_values:216 INFO now 288.2  avg 10.8  lo 2  hi 288  new or near max SCAI
2016-07-28 12:59 track_values:216 INFO now 298.7  avg 11.2  lo 2  hi 299  new or near max ACXM
2016-07-28 12:59 track_values:216 INFO now 378.5  avg 11.2  lo 2  hi 379  new or near max DNB
2016-07-28 12:59 track_values:216 INFO now 348.9  avg 10.9  lo 2  hi 349  new or near max UFPI
2016-07-28 12:59 track_values:216 INFO now 325.4  avg 11.0  lo 2  hi 325  new or near max UMBF
2016-07-28 12:59 track_values:216 INFO now 336.9  avg 11.3  lo 2  hi 337  new or near max ANIP
2016-07-28 12:59 track_values:216 INFO now 846.9  avg 11.7  lo 2  hi 847  new or near max PHYS
2016-07-28 12:59 track_values:216 INFO now 794.2  avg 12.4  lo 2  hi 794  new or near max HUBS
2016-08-08 12:59 track_values:216 INFO now 839.5  avg 11.6  lo 2  hi 840  new or near max DSL
2016-08-09 12:59 track_values:216 INFO now 756.4  avg 11.7  lo 1  hi 756  new or near max ORBC
2016-08-11 12:59 track_values:216 INFO now 1198.8  avg 11.8  lo 1  hi 1199  new or near max KRNY
2016-08-23 12:59 track_values:216 INFO now 1489.9  avg 12.7  lo 1  hi 1490  new or near max PHYS
2016-09-08 12:59 track_values:216 INFO now 2804.8  avg 12.8  lo 1  hi 2805  new or near max OUTR
2017-06-22 12:59 track_values:216 INFO now 6641.1  avg 13.6  lo 1  hi 6641  new or near max WMS <-- outlier

        volume_clipped[s] = volume[s].sort_values()[:int(len(volume) * .1)].fillna(0)
                                                                       ^^
2016-07-27 12:59 track_values:157 INFO now 17.604  avg 17.604  lo 18  hi 18  new max ARNC
2016-07-27 12:59 track_values:157 INFO now 27.579  avg 13.662  lo 4  hi 28  new max ADSK
2016-07-27 12:59 track_values:157 INFO now 124.580  avg 17.465  lo 4  hi 125  new max ALKS
2016-07-27 12:59 track_values:157 INFO now 233.805  avg 21.065  lo 4  hi 234  new max CUZ
2016-07-27 12:59 track_values:157 INFO now 252.681  avg 22.375  lo 4  hi 253  new max SEIC
2016-07-27 12:59 track_values:157 INFO now 495.768  avg 23.815  lo 4  hi 496  new max SLM
2016-07-29 12:59 track_values:157 INFO now 490.638  avg 30.071  lo 3  hi 491  new max SHO
2016-08-03 12:59 track_values:157 INFO now 541.886  avg 29.302  lo 3  hi 542  new max CENX
2016-08-03 12:59 track_values:157 INFO now 900.307  avg 29.648  lo 3  hi 900  new max FCAU
2016-10-31 12:59 track_values:157 INFO now 1068.872  avg 30.816  lo 2  hi 1069  new max BRCD
2016-11-04 12:59 track_values:157 INFO now 963.337  avg 30.734  lo 2  hi 963  new max ALR
2017-01-03 12:59 track_values:157 INFO now 1134.198  avg 30.146  lo 2  hi 1134  new max ACAS
2017-01-31 12:59 track_values:157 INFO now 1409.783  avg 30.471  lo 2  hi 1410  new max SRC

'''
from quantopian.pipeline  import Pipeline
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline.filters      import Q1500US
from quantopian.pipeline.experimental import QTradableStocksUS
import pandas as pd

def initialize(context):
    schedule_function(run, date_rules.every_day(), time_rules.market_close())
    attach_pipeline(Pipeline(
        #screen = Q1500US()
        screen = QTradableStocksUS()
    ), 'p')

def before_trading_start(context, data):
    context.stocks = pipeline_output('p').index

def run(context, data):
    volume = data.history(context.stocks, 'volume', 390, '1m')  # All day volumes
    volume_clipped = pd.DataFrame()

    for s in volume.T.index:
        # Just the bottom half values. Can this be done for all ahead of the loop instead?
        volume_clipped[s] = volume[s].sort_values()[:int(len(volume) * .5)].fillna(0)

        mean_clp  = volume_clipped[s].mean()     # Avg bottom half low volume
        if not mean_clp: continue
        if mean_clp != mean_clp: continue        # nan
        mean_all  = volume[s].mean()             # Avg of all
        ratio     = mean_all / mean_clp
        ratio_max = track_values(context, ratio, s)
        if 0 and ratio_max:
            log.info('mean_all {}  mean_clipped {}  ratio {}  {}'.format(
                '%.1f'%mean_all, '%.1f'%mean_clp, '%.0f'%(ratio), s.symbol))

def track_values(c, var, s):
    ''' https://www.quantopian.com/posts/track-values                 ** modified **

        Log current, average, high and low for a particular value.
        Incoming 'c' for brevity is context

        track_values(context, <THE VARIABLE TO BE TRACKED> )
    '''
    #log_level = 1  #  0   Off
                   #  1   Each new minimum or maximum
                   #  2   Everything always
    if 'val' not in c:
        c.val = {
            'hi' : var,
            'lo' : var,
            'avg': [0, 0],
        }
    avg, num     = c.val['avg']
    avg_new      = ((avg * num) + var) / (num + 1)
    c.val['avg'] = [avg_new, num + 1]
    new_str = ''
    if   var > c.val['hi']               * .9:   # catch some even if just near latest maximum
        c.val['hi'] = var
        new_str = 'new or near max'
    elif var < c.val['lo']:
        c.val['lo'] = var
        new_str = 'new min'
    #if new_str: # or log_level == 2:
    if new_str == 'new or near max': # or log_level == 2:
        log.info('now {}  avg {}  lo {}  hi {}  {} {}'.format(
            '%.1f' % var, '%.1f' % avg_new, '%.0f' % c.val['lo'], '%.0f' % c.val['hi'], new_str, s.symbol))
        return new_str


There was a runtime error.