TorchTrade

TorchTrade Documentation¶

Welcome to the TorchTrade documentation! TorchTrade is a machine learning framework for algorithmic trading built on TorchRL.

TorchTrade's goal is to provide accessible deployment of RL methods to trading. The framework supports various RL methodologies including online RL, offline RL, model-based RL, contrastive learning, and many more areas of reinforcement learning research. Beyond RL, TorchTrade integrates traditional trading methods such as rule-based strategies, as well as modern approaches including LLMs (both local models and frontier model integrations) as trading actors.

What is TorchTrade?¶

TorchTrade provides modular environments for both live trading with major exchanges and offline backtesting. The framework supports:

🎯 Multi-Timeframe Observations - Train on 1m, 5m, 15m, 1h bars simultaneously
🤖 Multiple RL Algorithms - PPO, DQN, IQL, GRPO, DSAC implementations
📊 Feature Engineering - Add technical indicators and custom features
🔴 Live Trading - Direct API integration with major exchanges
📉 Risk Management - Stop-loss/take-profit, margin, leverage, liquidation mechanics
🔮 Futures Trading - Up to 125x leverage with proper margin management
📦 Ready-to-Use Datasets - Pre-processed OHLCV data available at HuggingFace/Torch-Trade

Getting Started¶

Installation & Setup - Get up and running in minutes
First Environment - Create and run your first trading environment
First Training Run - Train a PPO policy

Environments¶

Offline Environments - Backtesting with historical data
- SequentialTradingEnv, SequentialTradingEnvSLTP, OneStepTradingEnv
Online Environments - Live trading with exchange APIs
- Alpaca, Binance, Bitget integrations

Components¶

Loss Functions - Training objectives (GRPOLoss, CTRLLoss, CTRLPPOLoss)
Transforms - Data preprocessing (CoverageTracker, ChronosEmbeddingTransform)
Actors - Trading policies (RuleBasedActor, FrontierLLMActor, LocalLLMActor)

Advanced Customization¶

Feature Engineering - Add technical indicators and features
Reward Functions - Design reward functions for your strategy
Performance Metrics - Evaluate and customize trading performance metrics

Key Features¶

Multi-Timeframe Support¶

Observe market data at multiple time scales simultaneously:

config = SequentialTradingEnvConfig(
    time_frames=["1min", "5min", "15min", "60min"],
    window_sizes=[12, 8, 8, 24],       # Lookback per timeframe
    execute_on=(5, "Minute")           # Execute every 5 minutes
)

Futures Trading with Leverage¶

Trade with leverage and manage margin:

config = SequentialTradingEnvConfig(
    leverage=10,                       # 10x leverage
    initial_cash=10000,
    margin_call_threshold=0.2,         # 20% margin ratio triggers liquidation
)

Stop-Loss / Take-Profit Bracket Orders¶

Risk management with combinatorial action spaces:

config = SequentialTradingEnvSLTPConfig(
    stoploss_levels=[-0.02, -0.05],    # -2%, -5%
    takeprofit_levels=[0.05, 0.10],    # +5%, +10%
    include_hold_action=True,          # Optional: set False to remove HOLD
)
# Action space: HOLD + (2 SL × 2 TP) = 5 actions (or 4 without HOLD)

Environment Comparison¶

Offline Environments (Backtesting)¶

All environments support both spot and futures trading via config (leverage=1 for spot, leverage>1 for futures with margin/liquidation mechanics).

Environment	Bracket Orders	One-Step	Best For
SequentialTradingEnv	❌	❌	Standard sequential trading
SequentialTradingEnvSLTP	✅	❌	Risk management with SL/TP
OneStepTradingEnv	✅	✅	GRPO, contextual bandits

Live Environments (Exchange APIs)¶

Environment	Exchange	Futures	Leverage	Bracket Orders
AlpacaTorchTradingEnv	Alpaca	❌	❌	❌
AlpacaSLTPTorchTradingEnv	Alpaca	❌	❌	✅
BinanceFuturesTorchTradingEnv	Binance	✅	✅	❌
BinanceFuturesSLTPTorchTradingEnv	Binance	✅	✅	✅
BitgetFuturesTorchTradingEnv	Bitget	✅	✅	❌
BitgetFuturesSLTPTorchTradingEnv	Bitget	✅	✅	✅

Next Steps¶

Getting Started Guide - Install TorchTrade and run your first environment.