Stock trades + quotes / seven steps
A week of data.
Your first replay.
Install the package, download recent stock data, and put it to work. Read the compressed CSV files immediately, then build per-symbol databases for replay and timestamp lookup.
Install the package
Create a virtual environment and install the Python package and its command-line tools.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install massive-speedup
If pip needs to build the native extension, you will also need a C++23 compiler, CMake 3.24+, and Git. See the installation guide for source-build instructions.
Set your keys and folders
From your Massive dashboard, copy your flatfile S3 access key and secret, plus your separate REST API key. Replace the three placeholders below and choose where the downloaded files and databases should live.
export AWS_ACCESS_KEY_ID="your-massive-flatfile-access-key"
export AWS_SECRET_ACCESS_KEY="your-massive-flatfile-secret-key"
export MASSIVE_API_KEY="your-massive-rest-api-key"
export MASSIVE_SPEEDUP_DOWNLOAD_PATH="$HOME/market-data/flatfiles"
export MASSIVE_SPEEDUP_DB_PATH="$HOME/market-data/databases"
The downloader uses the REST key to check data access and the S3 keys to fetch files. Both tools create their target folders as needed; Python database readers use the same database path automatically.
Download the past week
Calculate a cutoff from today and keep it for the database build. This downloads available stock trade and quote files from seven days ago through yesterday, inclusive.
export MS_WEEK_START=$(python -c \
'from datetime import date, timedelta; print(date.today() - timedelta(days=7))')
massive-speedup-download --products stocks --end-date "$MS_WEEK_START"
--end-date means the oldest included date:
the downloader works backward from yesterday. This is seven calendar
days, with files only where data is available; weekends and holidays
do not create extra trading sessions.
Files arrive in $MASSIVE_SPEEDUP_DOWNLOAD_PATH/stock_trade/
and stock_quote/, named YYYY-MM-DD.csv.gz.
These are market-wide files, so stock quotes can take substantial
disk space and download time. The examples select AAPL locally.
Existing downloads are skipped when you rerun the command.
Get a VWAP from the CSV
After the download finishes, read the newest session directly from
.csv.gz. This computes AAPL's volume-weighted average
price: total traded value divided by total reported share volume.
No extraction or database is needed.
import os
from pathlib import Path
import massive_speedup as ms
root = Path(os.environ["MASSIVE_SPEEDUP_DOWNLOAD_PATH"])
path = sorted(root.glob("stock_trade/*.csv.gz"))[-1]
notional = volume = 0
for trade in ms.FlatFiles.Stock.Trade.parse(path):
if trade.ticker == "AAPL":
notional += trade.price * trade.size
volume += trade.size
print(path.name[:10], "AAPL VWAP:", notional / volume if volume else None)
Each row has named fields such as ticker,
price, size, and sip_timestamp.
This example includes all reported AAPL trades and produces an
end-of-session statistic; do not feed that completed value into an
earlier decision. For quotes, use
ms.FlatFiles.Stock.Quote.parse(path) on a
stock_quote archive and read bid_price
and ask_price.
Build that week's databases
Use the same cutoff to convert the downloaded stock trades and quotes into per-symbol files, sorted by SIP timestamp. Run this in the terminal where you set the environment variables.
massive-speedup-build-database --only stock --not-before "$MS_WEEK_START"
For each downloaded session, AAPL's files will be at
$MASSIVE_SPEEDUP_DB_PATH/stock_trade/YYYY-MM-DD/AAPL
and stock_quote/YYYY-MM-DD/AAPL. Completed database
sessions are skipped on reruns. Wait for the build to finish before
continuing; you can keep using the CSV example while it runs.
Replay a session with the broker
Simulate buying and holding one AAPL share on the latest built day.
SimpleMarket iterates the trade and quote databases in
SIP timestamp order and supplies a broker. By default, the loop
receives trade events; quotes still update the simulated market.
import os
from pathlib import Path
import massive_speedup as ms
root = Path(os.environ["MASSIVE_SPEEDUP_DB_PATH"])
day = sorted(root.glob("stock_trade/*/AAPL"))[-1].parent.name
market = ms.SimpleMarket(day, ["AAPL"], trade_latency_ns=150_000_000)
for symbol, timestamp, trade, quote, trades, quotes, broker in market:
broker.set_desired(1)
print(market.summary())
set_desired(1) targets one share in total, so calling it
on each event does not keep buying more shares. Market buys use the
ask and sells use the bid, with the configured 150 ms latency.
The summary reports fills, cash, and positions; this example leaves
the share open at the end of the session.
Order submission returns no fill price.
market.summary() is available only after the iterator
is exhausted, keeping future execution prices out of this decision
loop. Replace the fixed target with a signal updated from each
incoming event, the same incremental pattern used by
RTTA.
Find the spread at market open
Jump directly to the last quote known at the opening bell.
market_open comes from the session calendar, so there
is no hard-coded UTC offset. The search uses SIP timestamps in
nanoseconds since the epoch.
import os
from pathlib import Path
import massive_speedup as ms
root = Path(os.environ["MASSIVE_SPEEDUP_DB_PATH"])
day = sorted(root.glob("stock_quote/*/AAPL"))[-1].parent.name
quotes = ms.StockQuoteDatabase(day, "AAPL")
i = quotes.index_before_timestamp(quotes.market_open)
if i >= 0:
quote = quotes[i]
print("Opening spread:", quote.ask_price - quote.bid_price)
else:
print("No quote at or before market open.")
index_before_timestamp(t) finds the final record at or
before t. It returns -1 if none exists,
so check before indexing: quotes[-1] would read the
final quote of the day. This lookup uses only information known
by the requested instant.