Inference-time scaling on Red Hat AI: Improving model reliability
How inference-time scaling techniques on Red Hat AI improve model reliability by letting models spend more compute at generation time on harder problems.
Open-source AI
A collective of researchers and engineers from Red Hat & IBM building LLM toolkits you can use today.
LLM Hubs
its_hub
Inference-time scaling for LLMs.
sdg_hub
Synthetic data generation pipelines
training_hub
Post training algorithms for LLMs
LLM Tools
async-grpo
Asynchronous GRPO for scalable reinforcement learning.
hopscotch
A method for skipping redundant attention blocks in language models
mini_trainer
Efficient training library for large language models up to 70B parameters on a single node.
orthogonal-subspace-learning
Adaptive SVD-based continual learning method for LLMs.
probabilistic-inference-scaling
Inference-time scaling with particle filtering.
reward_hub
State-of-the-art reward models for preference data generation and acceptance criteria.
SQuat
KV cache quantization for scaling inference time
training
Efficient messages-format SFT library for language models
Recent posts
View all posts
Particle filtering for better LLM reasoning: a different approach to inference-time scaling
Particle filtering offers a principled alternative to majority voting and beam search for inference-time scaling, improving LLM reasoning by pruning unpromising chains of thought early.
OSFT explained: Prevent catastrophic forgetting in LLM fine-tuning
Orthogonal Subspace Fine-Tuning (OSFT) prevents catastrophic forgetting during LLM fine-tuning by constraining weight updates to directions that preserve existing model capabilities.
Recent videos
View all videos📹 Scaling Training and Eval Data for Coding Agents
👤 Speaker: Ari Aye
August 07, 2026
📹 Policy-Driven Agentic Red-teaming
👤 Speaker: Muneeza Azmat
July 31, 2026
📹 Flexible Agent Alignment with Goal Inference from Open-Ended Dialog
👤 Speaker: Rachel Ma
July 17, 2026