AI Quantitative Trading: From Models to Quant Funds

1. Background Introduction

This is a 7-minute read where I’ll demystify traditional quantitative investing and introduce AI-powered quantitative models.

As a large language model expert, the industry acceleration driven by Scaling Law in 2023–2024 has deepened my understanding of machine learning’s essence and limitations. Confidently, I place myself in the top tier for both academic theory and model development. When top AI firms like ByteDance offered high salaries, I declined, believing the current approach of probability statistics + neural network function fitting has neared its ceiling (my exact words to them). While large models showcase remarkable capabilities through extensive engineering, their outputs—each word in text or pixel in images—are maximum likelihood approximations based on training data probabilities, limiting their ability to achieve optimal results or high accuracy for complex tasks. This creates challenges for enterprise applications, robotics, and autonomous driving, where progress remains exploratory.

AI is undeniably the hottest field, attracting talent and capital. However, many pursue entrepreneurship or innovation for its own sake. Blindly launching AI startups can be grueling, as Zhu Xiaohu noted: it’s about grinding through “dirty work” to build moats, with engineering optimization outweighing model breakthroughs. Even leading firms like OpenAI, since releasing the o1 Test-time compute model in September 2024, have shifted focus to product iteration—browser integration, deep search, Ghibli-style outputs, and Canvas—building moats through engineering rather than model innovation. Companies like Kimi are also investing heavily in research but haven’t yet surpassed Transformer’s diminishing returns, prompting leaders like Li Kaifu and Wang Xiaochuan to pivot their AI ventures.

If top-tier model architects number 100, then engineers number 10,000. For those 100, the paths are to research next-generation architectures beyond Transformers, overhaul simplistic backpropagation, or compete in engineering’s relentless grind—exhausting for seasoned professionals. So, are there better ways to seize the opportunities unleashed by Scaling Law?

The answer is yes. For non-intelligent vertical domains, this wave of AI model architectures is highly valuable because they don’t require real-world logical interaction, only judging outcomes based on feature values to select optimal strategies. A prime example is stock and options trading, where AI quantization excels with reinforcement learning models.

2. What is Traditional Quantitative Trading?

My prior exposure to stock investing was limited, though I’ve followed U.S. markets for 7–8 years, recommending friends invest in Tesla and Nvidia at lower prices. Those decisions stemmed from industry observations, typical of long-term holding, distinct from traditional quantization. Today, I’ll show how it connects deeply with AI quantization.

This isn’t a lengthy blog but an introductory article for my upcoming video series, AI Quantitative Trading: From Models to Quant Funds. With over 100,000 AI industry followers across platforms, we’ve covered Transformers to reinforcement learning and now dive into AI quantitative investing. Unlike Huatai’s journey from quantization to launching DeepSeek, we’re reversing the path. Some may think quantization is complex and fraught with pitfalls, but entry can be surprisingly swift—its core is far simpler than large language models!

Quant enthusiasts often hear: this field is filled with math geniuses obsessed with algorithms. This intimidates newcomers, but after diving in over the past month, I can responsibly clarify:

• The “math” involves decades-old indicators and fixed formulas, now encapsulated in programming libraries—no need to derive them yourself.
• Quantitative strategies entail tweaking and recombining these formula parameters, then testing on historical stock data.
• Quant strategy models are far less complex to train than billion-parameter LLMs or diffusion models.
• Current stock “Agents” are rudimentary, like feeding a stock’s OHLCV data and news to models (e.g., GPT-4o, DeepSeek) for investment advice—unreliable for serious funds.

I’ll elaborate on these in my video series and open-source GitHub. The last point is particularly flawed: if throwing data at large models worked, funds would already dominate. These models aren’t trained with stock-specific datasets or loss functions, making their decisions untrustworthy for professional use.

Why, then, do quant funds seem so mysterious? First, high entry barriers: startup capital, licenses, industry experience, and connections. Most can’t launch their own fund, so they join pooled investments (e.g., Huatai’s 5M RMB minimum) or trade independently on quant platforms. Second, why are firms like Huatai, Jane Street, and Citadel so profitable? Key factors include:

• High-Frequency Trading (HFT): Relies on speed (e.g., co-locating servers with exchanges) and market-making rebates, not complex AI.
• High Volatility: Profits soar in volatile markets.
• Market Trends: Accurate fundamental predictions in trending markets (e.g., U.S. stocks’ 5–6-year bull run) yield high returns.
• Early-Mover Advantage: Early quant funds outpaced less tech-savvy institutions and retail investors, though competition now intensifies (e.g., Huatai’s 2024 underperformance).
• Fast Reactions: Jane Street’s 2024 dominance in India’s options market, capturing ~70% of profits via HFT, is a prime example.

Jane Street’s 2024 revenue doubled, surpassing Morgan Stanley, largely due to AI-driven trading. This reflects the industry’s shift from traditional to AI quantization. Why did firms like Huatai and Jane Street invest heavily in GPUs pre-AI boom? Likely for:

1. Training and backtesting on massive minute/microsecond-level historical data.
2. Analyzing news, sentiment, and corporate filings.
3. Comparing model architectures (e.g., LSTM, XGBoost, to recent Diffusion, Transformers).

These technical foundations positioned AI quantization for a leap forward with the large model surge.

In summary, quant investing’s core is accessible to AI experts. The mystery lies not in intellectual barriers but in information gaps.

3. How to Implement AI Quantitative Trading?

Traditional quantization was outlined above. Even if not fully clear, this encapsulates it:

Quantitative strategy is essentially fitting a complex function.

This function’s parameters are indicator combinations, outputting buy/sell actions likely to profit. Quantization seeks optimal parameter nesting and combinations.

Traditional methods apply human-defined indicators to assets, backtesting for high returns. If results falter, strategies grow more complex with added factors. Yet, even top strategies often yield mixed results, sometimes losing money. Take Medallion Fund, founded by James Simons in 1988, a mathematical modeling benchmark. It’s consistently profitable, yet in Q1 2025, it reportedly incurred losses. This highlights a core issue: human-defined strategy parameters are a finite set. They may excel in historical scenarios but fail in uncovered edge cases, like rare market events.

The solution? Deep neural networks, particularly reinforcement learning (RL), are ideal for stock trading:

• Features: OHLCV (Open, High, Low, Close, Volume), technical indicators (e.g., RSI), or sentiment (e.g., news, filings).
• Actions: Long or Short (buy/sell).
• Rewards: 1, -1 (profit/loss).

This binary outcome suits large-scale RL, theoretically fitting a highly complex strategy function. Compare 100 human-set parameters to a billion-parameter model—which better captures complex probability distributions? We needn’t understand the neurons’ interactions; they’re vastly more intricate than manual parameters.

However, training robust AI quant models demands deep expertise in model design and mechanics. While papers on stock models or multi-agent systems exist, their benchmarks (e.g., S&P 500, 2020–2024) often cover bull markets where basic strategies outperform. In volatile or bearish cycles, their performance requires scrutiny. Most open-source strategies fall short of fund-grade returns and risk controls, often serving academic purposes.

In my upcoming open-source work, I’ll address these gaps (partially), raising key considerations:

• Larger, richer datasets are better, akin to GPT’s scaling.
• Beyond static indicators, incorporate human expertise (e.g., learning top traders’ strategies).
• Hedging to mitigate risks in dynamic markets.
• Capital management, dynamically adjusting trade sizes via a trained model.
• One model per asset (or portfolio), not a universal AI for all trades.
• Financial data is abundant, but noise filtering is critical.

Models vs. Human Brains: Models lack human logical finesse but excel in repetitive, long-term pattern recall by orders of magnitude. By learning historical patterns and continuously updating with trader strategies, models achieve a dynamic, half-human, half-machine AI state.

4. The Real Barriers to AI Quantitative Trading

Practical implementation isn’t trivial. Beyond strategy decay, data curation, low-latency execution, large-scale training, and multi-asset coverage, here are high-level barriers:

• HFT: The backbone of firms like Jane Street’s profits. Speed hinges on server proximity, processor performance, code efficiency, and microsecond-level network latency.
• Trading Counterparties: U.S. markets are ~80% institutional, 20% retail. Profitable quant funds often profit from other institutions’ losses, competing on order speed and algorithmic edge.
• Experience: Decades-old industries have an insurmountable barrier—practitioner experience. Real trading’s psychological threshold is stark: novices and veteran traders judge asset movements differently. Yet, AI offers newcomers a chance: a sufficiently complex, emotionless AI can outperform human decisions in many scenarios.
• Capital Management: A mediocre model with savvy capital allocation outperforms a brilliant model without it over a year. Capital management itself requires a deeply trained AI.

Even with AI quant tools, decisions aren’t foolproof. Machines can’t counter insider trading, though this lies outside our scope. Models can factor in such risks, albeit imperfectly. For instance, during Trump’s 2024 trade war, my AI, monitoring Truth Social and X channels, predicted a bearish trend. However, the rebound after Trump’s tariff rollback was harder for algorithms to pinpoint. This shows human judgment often outshines algorithms in rare events, but their synergy prevails 🏆.

5. Join Me in Practicing AI Quantization

I’m LLM Teacher, active on Bilibili, Douyin, and WeChat Channels. A survey of my AI followers showed ~15% are highly interested in AI quantization. As quant-experienced fund managers may also follow, I plan a comprehensive, beginner-friendly journey to build and trade AI quant models. This AI Quantitative Trading: From Models to Quant Funds series blends learning and hands-on projects, covering concepts, indicator math, a stock (later options, ETFs, crypto) strategy backtesting and signal platform, and training AI quant Agents for real trades.